2020-12-13T00:00:51.713 INFO:root:teuthology version: 1.0.0-a4989a90
2020-12-13T00:00:51.714 INFO:teuthology.results:subset: '10/9999'
2020-12-13T00:00:51.715 INFO:teuthology.results:seed: '3484'
2020-12-13T00:16:32.489 INFO:root:teuthology version: 1.0.0-a4989a90
2020-12-13T00:16:32.522 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-12-13T12:13:45.254 INFO:teuthology.results:Tests finished! gathering results...
2020-12-13T12:13:48.137 INFO:scrape:Found 24 jobs
2020-12-13T12:13:48.993 INFO:scrape:Found 2 distinct failure reasons
2020-12-13T12:13:48.994 INFO:scrape:Dead: b'2020-12-13T12:10:58.738 DEBUG:teuthology.exit:Finished running handlers'
2020-12-13T12:13:48.994 INFO:scrape:1 jobs: ['5696379']
2020-12-13T12:13:48.995 INFO:scrape:suites: ['begin', 'ceph-thrash/mds', 'clusters/3-mds-2-standby', 'conf/{client', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multimds/thrash/{0-supported-random-distro$/{rhel_latest}', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{fuse-default-perm-no', 'session_timeout', 'tasks/cfuse_workunit_suites_pjd}', 'thrash/{frag_enable', 'thrash_debug}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-12-13T12:13:48.995 INFO:scrape:
2020-12-13T12:13:48.996 INFO:scrape:Failure: 
2020-12-13T12:13:48.996 INFO:scrape:2 jobs: ['5696378', '5696370']
2020-12-13T12:13:48.996 INFO:scrape:suites intersection: ['centos_latest', 'clusters/3-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'multimds/verify/{begin', 'osd}', 'overrides/{fuse-default-perm-no', 'session_timeout', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-12-13T12:13:48.997 INFO:scrape:suites union: ['centos_latest', 'clusters/3-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'multimds/verify/{begin', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{fuse-default-perm-no', 'session_timeout', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-12-13T12:13:48.997 INFO:scrape:
2020-12-13T12:13:48.999 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/
failed:  2
dead:    1
running: 0
waiting: 0
queued:  0
passed:  21


Fail
=================================================================
[5696370]  multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   03:42:10
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696370/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696370/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2571348f1cd7473faf8c4a7747d0b363

[5696378]  multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   03:25:47
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696378/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696378/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8bfcc09779cb4c9f9e42f19fc09b0ce3



Dead
=================================================================
[5696379]  multimds/thrash/{0-supported-random-distro$/{rhel_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696379/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696379/



Pass
=================================================================
[5696369] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:27:01
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696369/

[5696371] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:17:47
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696371/

[5696372] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:17:11
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696372/

[5696373] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:42:34
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696373/

[5696374] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:05:27
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696374/

[5696375] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:21:41
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696375/

[5696376] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:26:26
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696376/

[5696377] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:00:56
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696377/

[5696380] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   00:51:41
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696380/

[5696381] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:21:41
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696381/

[5696382] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:00:35
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696382/

[5696383] multimds/thrash/{0-supported-random-distro$/{rhel_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:21
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696383/

[5696384] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:41:47
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696384/

[5696385] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   00:58:07
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696385/

[5696386] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
time:   02:44:19
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696386/

[5696387] multimds/thrash/{0-supported-random-distro$/{centos_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:17:52
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696387/

[5696388] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   01:12:46
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696388/

[5696389] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:24:13
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696389/

[5696390] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:05:19
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696390/

[5696391] multimds/thrash/{0-supported-random-distro$/{rhel_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:21:40
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696391/

[5696392] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:30:50
info:   http://pulpito.ceph.com/teuthology-2020-12-09_13:15:04-multimds-octopus-distro-basic-smithi/5696392/