2020-06-04T10:46:40.150 INFO:root:teuthology version: 1.0.0-c66e34b 2020-06-04T10:46:40.154 INFO:teuthology.results:subset: '16/999' 2020-06-04T10:46:40.154 INFO:teuthology.results:seed: '1726' 2020-06-04T11:02:25.129 INFO:root:teuthology version: 1.0.0-c66e34b 2020-06-04T11:02:25.150 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-06-04T15:52:31.659 INFO:teuthology.results:Tests finished! gathering results... 2020-06-04T15:52:31.717 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 22 Fail ================================================================= [5115972] multimds/verify/{begin clusters/9-mds conf/{client mds mon osd} mount/kclient objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind} ----------------------------------------------------------------- time: 00:57:27 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115972/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115972/ "2020-06-04 11:48:52.469353 mon.b mon.0 172.21.15.112:6789/0 312 : cluster [WRN] Health check failed: 1 clients failing to respond to capability release (MDS_CLIENT_LATE_RELEASE)" in cluster log [5115980] multimds/verify/{begin clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind} ----------------------------------------------------------------- time: 04:34:24 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115980/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115980/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=145388d9a7d14338a88dbda5414a5586 Pass ================================================================= [5115958] multimds/basic/{begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter supported-random-distros$/{rhel_latest} tasks/cfuse_workunit_suites_blogbench} time: 00:29:26 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115958/ [5115959] multimds/thrash/{begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} supported-random-distros$/{ubuntu_16.04} tasks/cfuse_workunit_suites_fsstress} time: 00:29:32 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115959/ [5115960] multimds/verify/{begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep} time: 00:28:41 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115960/ [5115961] multimds/basic/{begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter supported-random-distros$/{ubuntu_latest} tasks/cfuse_workunit_suites_dbench} time: 01:26:17 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115961/ [5115962] multimds/basic/{begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter supported-random-distros$/{ubuntu_16.04} tasks/cfuse_workunit_suites_ffsb} time: 00:35:06 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115962/ [5115963] multimds/thrash/{begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} supported-random-distros$/{ubuntu_latest} tasks/cfuse_workunit_suites_pjd} time: 00:14:39 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115963/ [5115964] multimds/verify/{begin clusters/9-mds conf/{client mds mon osd} mount/kclient objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind} time: 00:54:26 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115964/ [5115965] multimds/basic/{begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter supported-random-distros$/{ubuntu_latest} tasks/cfuse_workunit_suites_fsstress} time: 00:20:43 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115965/ [5115966] multimds/basic/{begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter supported-random-distros$/{rhel_latest} tasks/cfuse_workunit_suites_fsx} time: 01:47:47 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115966/ [5115967] multimds/thrash/{begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} supported-random-distros$/{rhel_latest} tasks/cfuse_workunit_suites_fsstress} time: 00:31:42 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115967/ [5115968] multimds/verify/{begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep} time: 00:28:45 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115968/ [5115969] multimds/basic/{begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter supported-random-distros$/{ubuntu_latest} tasks/cfuse_workunit_suites_pjd} time: 00:14:11 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115969/ [5115970] multimds/basic/{begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter supported-random-distros$/{ubuntu_latest} tasks/cephfs_test_exports} time: 00:11:51 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115970/ [5115971] multimds/thrash/{begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} supported-random-distros$/{ubuntu_16.04} tasks/cfuse_workunit_suites_pjd} time: 00:13:57 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115971/ [5115973] multimds/basic/{begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter supported-random-distros$/{ubuntu_latest} tasks/cephfs_test_snapshots} time: 00:20:05 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115973/ [5115974] multimds/basic/{begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter supported-random-distros$/{rhel_latest} tasks/cfuse_workunit_kernel_untar_build} time: 03:06:59 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115974/ [5115975] multimds/thrash/{begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} supported-random-distros$/{ubuntu_16.04} tasks/cfuse_workunit_suites_fsstress} time: 00:17:33 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115975/ [5115976] multimds/verify/{begin clusters/3-mds conf/{client mds mon osd} mount/kclient objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep} time: 01:26:33 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115976/ [5115977] multimds/basic/{begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter supported-random-distros$/{ubuntu_latest} tasks/cfuse_workunit_misc} time: 01:43:33 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115977/ [5115978] multimds/basic/{begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter supported-random-distros$/{ubuntu_latest} tasks/cfuse_workunit_norstats} time: 00:18:55 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115978/ [5115979] multimds/thrash/{begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} supported-random-distros$/{ubuntu_latest} tasks/cfuse_workunit_suites_pjd} time: 00:14:55 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115979/ [5115981] multimds/basic/{begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter supported-random-distros$/{ubuntu_16.04} tasks/cfuse_workunit_suites_blogbench} time: 00:41:27 info: http://pulpito.ceph.com/teuthology-2020-06-03_14:15:03-multimds-mimic-testing-basic-smithi/5115981/