2020-11-16T06:39:28.943 INFO:root:teuthology version: 1.0.0-357fd4a
2020-11-16T06:39:28.946 INFO:teuthology.results:subset: '14/100000'
2020-11-16T06:39:28.947 INFO:teuthology.results:seed: '8966'
2020-11-16T14:20:17.239 INFO:root:teuthology version: 1.0.0-357fd4a
2020-11-16T14:20:17.274 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-11-16T17:35:20.788 INFO:teuthology.results:Tests finished! gathering results...
2020-11-16T17:35:24.731 INFO:scrape:Found 25 jobs
2020-11-16T17:35:25.078 WARNING:scrape:Misunderstood line: Binary file /home/teuthworker/archive/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628641/teuthology.log matches
2020-11-16T17:35:25.079 WARNING:scrape:Misunderstood line: 
2020-11-16T17:35:25.362 INFO:scrape:Found 1 distinct failure reasons
2020-11-16T17:35:25.363 INFO:scrape:Valgrind: 
2020-11-16T17:35:25.363 INFO:scrape:2 jobs: ['5628641', '5628623']
2020-11-16T17:35:25.363 INFO:scrape:suites intersection: ['clusters/9-mds', 'conf/{client', 'mds', 'mon', 'osd}', 'session_timeout', 'whitelist_health']
2020-11-16T17:35:25.365 INFO:scrape:suites union: ['begin', 'centos_latest', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'mds', 'mon', 'mon-debug', 'mount', 'mount/fuse', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'multimds/basic/{0-supported-random-distro$/{rhel_8}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'osd-asserts', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{fuse-default-perm-no', 'q_check_counter/check_counter', 'session_timeout', 'tasks/cfuse_workunit_kernel_untar_build}', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'whitelist_wrongly_marked_down}}']
2020-11-16T17:35:25.365 INFO:scrape:
2020-11-16T17:35:25.367 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/
failed:  2
dead:    0
running: 0
waiting: 0
queued:  0
passed:  23


Fail
=================================================================
[5628623]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
-----------------------------------------------------------------
time:   02:29:15
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628623/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628623/
sentry: https://sentry.ceph.com/organizations/ceph/?query=38047f50f3874ce4a723dd8e384550a9

    Command failed on smithi142 with status 1: 'sudo rm -rf --
    /home/ubuntu/cephtest/mnt.1/client.1/tmp'

[5628641]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   02:06:44
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628641/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628641/

    saw valgrind issues



Pass
=================================================================
[5628622] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:15:04
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628622/

[5628624] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   00:46:42
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628624/

[5628625] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
time:   02:37:56
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628625/

[5628626] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:25:10
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628626/

[5628627] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:16:06
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628627/

[5628628] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:43:20
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628628/

[5628629] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   02:01:54
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628629/

[5628630] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:46
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628630/

[5628631] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:11:34
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628631/

[5628632] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   02:07:16
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628632/

[5628633] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
time:   03:01:30
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628633/

[5628634] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:22:38
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628634/

[5628635] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:17:20
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628635/

[5628636] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:29:01
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628636/

[5628637] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:46:35
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628637/

[5628638] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:17:35
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628638/

[5628639] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:14:03
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628639/

[5628640] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:40:46
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628640/

[5628642] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:18:13
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628642/

[5628643] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:22:26
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628643/

[5628644] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:16:17
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628644/

[5628645] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:16:06
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628645/

[5628646] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   03:07:57
info:   http://pulpito.ceph.com/teuthology-2020-11-16_04:15:03-multimds-master-distro-basic-smithi/5628646/