2020-10-31T04:16:34.080 INFO:root:teuthology version: 1.0.0-f7de558
2020-10-31T04:16:34.082 INFO:teuthology.results:subset: '26/100000'
2020-10-31T04:16:34.083 INFO:teuthology.results:seed: '1077'
2020-10-31T04:43:52.501 INFO:root:teuthology version: 1.0.0-f7de558
2020-10-31T04:43:52.533 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-10-31T09:33:58.027 INFO:teuthology.results:Tests finished! gathering results...
2020-10-31T09:34:01.153 INFO:scrape:Found 25 jobs
2020-10-31T09:34:02.067 INFO:scrape:Found 1 distinct failure reasons
2020-10-31T09:34:02.068 INFO:scrape:Failure: 
2020-10-31T09:34:02.069 INFO:scrape:2 jobs: ['5577331', '5577323']
2020-10-31T09:34:02.069 INFO:scrape:suites intersection: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'multimds/verify/{begin', 'osd}', 'overrides/{fuse-default-perm-no', 'session_timeout', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-10-31T09:34:02.070 INFO:scrape:suites union: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{fuse-default-perm-no', 'session_timeout', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-10-31T09:34:02.070 INFO:scrape:
2020-10-31T09:34:02.073 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/
failed:  2
dead:    0
running: 0
waiting: 0
queued:  0
passed:  23


Fail
=================================================================
[5577323]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   04:14:11
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577323/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577323/
sentry: https://sentry.ceph.com/ceph/sepia/?query=3bce07a6ba534fdea317d93ce4d694b7

[5577331]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   03:32:17
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577331/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577331/
sentry: https://sentry.ceph.com/ceph/sepia/?query=eabc6400358248f4ac7ab82f924009fa



Pass
=================================================================
[5577309] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:17:24
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577309/

[5577310] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:51
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577310/

[5577311] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:22:33
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577311/

[5577312] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:18:19
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577312/

[5577313] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:04:52
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577313/

[5577314] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:16:58
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577314/

[5577315] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   00:56:36
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577315/

[5577316] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   00:55:42
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577316/

[5577317] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:20:27
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577317/

[5577318] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:49:04
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577318/

[5577319] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:02
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577319/

[5577320] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:45:30
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577320/

[5577321] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:39:01
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577321/

[5577322] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:32:33
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577322/

[5577324] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:16:04
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577324/

[5577325] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:19:29
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577325/

[5577326] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:51:50
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577326/

[5577327] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:46:28
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577327/

[5577328] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:16:56
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577328/

[5577329] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:34:14
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577329/

[5577330] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:39:54
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577330/

[5577332] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:46
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577332/

[5577333] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:16:15
info:   http://pulpito.ceph.com/teuthology-2020-10-31_04:15:03-multimds-master-distro-basic-smithi/5577333/