2020-11-28T04:16:28.849 INFO:root:teuthology version: 1.0.0-e0aaa9bf
2020-11-28T04:16:28.851 INFO:teuthology.results:subset: '26/100000'
2020-11-28T04:16:28.852 INFO:teuthology.results:seed: '8151'
2020-11-28T04:32:31.737 INFO:root:teuthology version: 1.0.0-e0aaa9bf
2020-11-28T04:32:31.761 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-11-28T09:13:36.978 INFO:teuthology.results:Tests finished! gathering results...
2020-11-28T09:13:40.205 INFO:scrape:Found 26 jobs
2020-11-28T09:13:40.946 INFO:scrape:Found 1 distinct failure reasons
2020-11-28T09:13:40.946 INFO:scrape:Failure: 
2020-11-28T09:13:40.947 INFO:scrape:2 jobs: ['5661679', '5661688']
2020-11-28T09:13:40.947 INFO:scrape:suites intersection: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'multimds/verify/{begin', 'osd}', 'overrides/{fuse-default-perm-no', 'session_timeout', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-11-28T09:13:40.948 INFO:scrape:suites union: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{fuse-default-perm-no', 'session_timeout', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-11-28T09:13:40.948 INFO:scrape:
2020-11-28T09:13:40.950 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/
failed:  2
dead:    0
running: 0
waiting: 0
queued:  0
passed:  24


Fail
=================================================================
[5661679]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   04:27:51
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661679/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661679/
sentry: https://sentry.ceph.com/organizations/ceph/?query=110381310ce94bdc98ae04e74d09718d

[5661688]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   03:50:19
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661688/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661688/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7df3e774afa3498c9643ac87cf6a14f4



Pass
=================================================================
[5661669] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:30:18
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661669/

[5661670] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:15:24
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661670/

[5661671] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:15:48
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661671/

[5661672] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   00:55:38
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661672/

[5661673] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:52:16
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661673/

[5661674] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:15:39
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661674/

[5661675] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:28:17
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661675/

[5661676] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:26:54
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661676/

[5661677] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}
time:   00:14:52
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661677/

[5661678] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:16:32
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661678/

[5661680] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:14:09
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661680/

[5661681] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   04:44:34
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661681/

[5661682] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   00:37:38
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661682/

[5661683] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:38:52
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661683/

[5661684] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:17:04
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661684/

[5661685] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:22:23
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661685/

[5661686] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:41:29
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661686/

[5661687] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:34:26
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661687/

[5661689] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:15:09
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661689/

[5661690] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   02:20:13
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661690/

[5661691] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:18:35
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661691/

[5661692] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   00:58:30
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661692/

[5661693] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:45
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661693/

[5661694] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:30:59
info:   http://pulpito.ceph.com/teuthology-2020-11-28_04:15:03-multimds-master-distro-basic-smithi/5661694/