2020-06-30T18:02:11.104 INFO:root:teuthology version: 1.0.0-645528e
2020-06-30T18:02:11.105 INFO:teuthology.results:subset: '15/9999'
2020-06-30T18:02:11.106 INFO:teuthology.results:seed: '3517'
2020-06-30T19:04:45.890 INFO:root:teuthology version: 1.0.0-645528e
2020-06-30T19:04:45.924 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-06-30T20:43:47.924 INFO:teuthology.results:Tests finished! gathering results...
2020-06-30T20:43:50.084 INFO:scrape:Found 24 jobs
2020-06-30T20:43:50.462 INFO:scrape:Found 1 distinct failure reasons
2020-06-30T20:43:50.464 INFO:scrape:Failure: "2020-06-30T18:30:03.673154+0000 mon.a (mon.0) 109 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log
2020-06-30T20:43:50.464 INFO:scrape:1 jobs: ['5189771']
2020-06-30T20:43:50.465 INFO:scrape:suites: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{fuse-default-perm-no', 'tasks/cfuse_workunit_suites_dbench', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-06-30T20:43:50.465 INFO:scrape:
2020-06-30T20:43:50.467 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/
failed:  1
dead:    0
running: 0
waiting: 0
queued:  0
passed:  23


Fail
=================================================================
[5189771]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
-----------------------------------------------------------------
time:   01:05:21
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189771/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189771/

    "2020-06-30T18:30:03.673154+0000 mon.a (mon.0) 109 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log



Pass
=================================================================
[5189773] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:08:31
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189773/

[5189775] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:19:53
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189775/

[5189777] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:26:14
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189777/

[5189779] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:24:39
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189779/

[5189781] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:47:12
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189781/

[5189783] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:18:00
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189783/

[5189785] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:03:33
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189785/

[5189787] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   00:58:54
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189787/

[5189789] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   01:32:10
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189789/

[5189791] multimds/thrash/{0-supported-random-distro$/{rhel_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:30:00
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189791/

[5189793] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:23:26
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189793/

[5189795] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:27:52
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189795/

[5189797] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:26:59
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189797/

[5189799] multimds/thrash/{0-supported-random-distro$/{centos_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:20:00
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189799/

[5189801] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   01:01:56
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189801/

[5189803] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:18:35
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189803/

[5189805] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:15:09
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189805/

[5189807] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:39
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189807/

[5189809] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:28:00
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189809/

[5189811] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:27:02
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189811/

[5189813] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   00:45:01
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189813/

[5189815] multimds/thrash/{0-supported-random-distro$/{centos_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:17:01
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189815/

[5189817] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   00:59:32
info:   http://pulpito.ceph.com/teuthology-2020-06-29_13:15:03-multimds-octopus-testing-basic-smithi/5189817/