2020-10-21T04:17:47.424 INFO:root:teuthology version: 1.0.0-b106d09
2020-10-21T04:17:47.426 INFO:teuthology.results:subset: '16/100000'
2020-10-21T04:17:47.426 INFO:teuthology.results:seed: '5435'
2020-10-21T08:21:36.714 INFO:root:teuthology version: 1.0.0-b106d09
2020-10-21T08:21:36.763 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-10-21T13:00:42.042 INFO:teuthology.results:Tests finished! gathering results...
2020-10-21T13:00:45.048 INFO:scrape:Found 25 jobs
2020-10-21T13:00:46.256 INFO:scrape:Found 2 distinct failure reasons
2020-10-21T13:00:46.257 INFO:scrape:Failure: reached maximum tries (50) after waiting for 300 seconds
2020-10-21T13:00:46.258 INFO:scrape:1 jobs: ['5543999']
2020-10-21T13:00:46.258 INFO:scrape:suites: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'multimds/verify/{begin', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{fuse-default-perm-no', 'tasks/cfuse_workunit_suites_dbench', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-10-21T13:00:46.258 INFO:scrape:
2020-10-21T13:00:46.259 INFO:scrape:Failure: 
2020-10-21T13:00:46.259 INFO:scrape:1 jobs: ['5544007']
2020-10-21T13:00:46.259 INFO:scrape:suites: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'multimds/verify/{begin', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{fuse-default-perm-no', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-10-21T13:00:46.260 INFO:scrape:
2020-10-21T13:00:46.261 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/
failed:  2
dead:    0
running: 0
waiting: 0
queued:  0
passed:  23


Fail
=================================================================
[5543999]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
-----------------------------------------------------------------
time:   01:00:10
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5543999/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5543999/

    reached maximum tries (50) after waiting for 300 seconds

[5544007]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   04:25:39
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544007/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544007/
sentry: https://sentry.ceph.com/sentry/sepia/?query=7f57d8d672784887b701226b788ba350



Pass
=================================================================
[5543990] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   02:57:27
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5543990/

[5543991] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   01:00:42
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5543991/

[5543992] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   02:43:37
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5543992/

[5543993] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:15:48
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5543993/

[5543994] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:51
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5543994/

[5543995] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:20:27
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5543995/

[5543996] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:34:55
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5543996/

[5543997] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:46:18
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5543997/

[5543998] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:15:19
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5543998/

[5544000] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:51:00
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544000/

[5544001] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:22:43
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544001/

[5544002] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:35
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544002/

[5544003] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:33:07
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544003/

[5544004] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:22:40
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544004/

[5544005] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:14:18
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544005/

[5544006] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:14:54
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544006/

[5544008] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:40:41
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544008/

[5544009] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:17:56
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544009/

[5544010] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:11:59
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544010/

[5544011] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:22:14
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544011/

[5544012] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:16:44
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544012/

[5544013] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   02:31:34
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544013/

[5544014] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:23:03
info:   http://pulpito.ceph.com/teuthology-2020-10-21_04:15:02-multimds-master-distro-basic-smithi/5544014/