2020-07-05T13:15:48.333 INFO:root:teuthology version: 1.0.0-2b31d02
2020-07-05T13:15:48.336 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-07-05T13:15:48.948 INFO:root:teuthology version: 1.0.0-2b31d02
2020-07-05T13:15:48.950 INFO:teuthology.results:subset: '27/2999'
2020-07-05T13:15:48.950 INFO:teuthology.results:seed: '8390'
2020-07-05T16:18:51.686 INFO:teuthology.results:Tests finished! gathering results...
2020-07-05T16:18:53.185 INFO:scrape:Found 24 jobs
2020-07-05T16:18:56.889 INFO:scrape:Found 5 distinct failure reasons
2020-07-05T16:18:56.890 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=kernel&flavor=default&distros=centos%2F7%2Fx86_64&sha1=a5269762c4b7bb3655b3e41168328fce85946603
2020-07-05T16:18:56.891 INFO:scrape:8 jobs: ['5199941', '5199929', '5199927', '5199943', '5199903', '5199915', '5199913', '5199917']
2020-07-05T16:18:56.891 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'mount/fuse', 'osd}', 'whitelist_health']
2020-07-05T16:18:56.892 INFO:scrape:suites union: ['begin', 'centos_latest', 'ceph-thrash/default', 'clusters/3-mds', 'clusters/3-mds-2-standby', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'inline/yes', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'multimds/basic/{0-supported-random-distro$/{centos_latest}', 'multimds/thrash/{0-supported-random-distro$/{centos_latest}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{fuse-default-perm-no', 'q_check_counter/check_counter', 'tasks/cephfs_test_exports}', 'tasks/cfuse_workunit_misc}', 'tasks/cfuse_workunit_suites_blogbench}', 'tasks/cfuse_workunit_suites_dbench', 'tasks/cfuse_workunit_suites_fsstress', 'tasks/cfuse_workunit_suites_fsstress}', 'thrash/{frag_enable', 'thrash_debug}', 'validater/lockdep}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'whitelist_wrongly_marked_down}}']
2020-07-05T16:18:56.892 INFO:scrape:
2020-07-05T16:18:56.892 INFO:scrape:Failure: "2020-07-05 15:03:08.640081 mon.b (mon.0) 759 : cluster [WRN] Health check failed: 1 clients failing to respond to capability release (MDS_CLIENT_LATE_RELEASE)" in cluster log
2020-07-05T16:18:56.893 INFO:scrape:1 jobs: ['5199921']
2020-07-05T16:18:56.893 INFO:scrape:suites: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{kernel-testing', 'ms-die-on-skipped}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{fuse-default-perm-no', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-07-05T16:18:56.894 INFO:scrape:
2020-07-05T16:18:56.894 INFO:scrape:Failure: '/home/ubuntu/cephtest/archive/syslog/kern.log:2020-07-05T13:48:27.359403+00:00 smithi132 kernel: [ 1437.528260] watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/3:1:28678]
' in syslog
2020-07-05T16:18:56.895 INFO:scrape:1 jobs: ['5199919']
2020-07-05T16:18:56.895 INFO:scrape:suites: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_16.04}', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/random/{k-testing', 'q_check_counter/check_counter', 'supported$/{ubuntu_latest}}', 'tasks/cfuse_workunit_suites_fsx}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-07-05T16:18:56.895 INFO:scrape:
2020-07-05T16:18:56.896 INFO:scrape:Failure: Command failed (workunit test suites/fsstress.sh) on smithi196 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=120f482407ab4070126cb7ffc2f4f1b37517b4a0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'
2020-07-05T16:18:56.896 INFO:scrape:1 jobs: ['5199931']
2020-07-05T16:18:56.897 INFO:scrape:suites: ['begin', 'ceph-thrash/default', 'clusters/3-mds-2-standby', 'conf/{client', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multimds/thrash/{0-supported-random-distro$/{ubuntu_latest}', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{distro/rhel/{k-distro', 'overrides/{fuse-default-perm-no', 'rhel_latest}', 'tasks/cfuse_workunit_suites_fsstress}', 'thrash/{frag_enable', 'thrash_debug}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-07-05T16:18:56.897 INFO:scrape:
2020-07-05T16:18:56.897 INFO:scrape:Failure: '/home/ubuntu/cephtest/archive/syslog/misc.log:2020-07-05T13:51:24.274119+00:00 smithi189 crond[42958]: (root) INFO (Job execution of per-minute job scheduled for 13:50 delayed into subsequent minute 13:51. Skipping job run.)
' in syslog
2020-07-05T16:18:56.898 INFO:scrape:1 jobs: ['5199911']
2020-07-05T16:18:56.898 INFO:scrape:suites: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_16.04}', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/rhel/{k-distro', 'q_check_counter/check_counter', 'rhel_latest}', 'tasks/cfuse_workunit_suites_ffsb}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-07-05T16:18:56.899 INFO:scrape:
2020-07-05T16:18:56.900 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/
failed:  12
dead:    0
running: 0
waiting: 0
queued:  0
passed:  12


Fail
=================================================================
[5199903]  multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199903/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199903/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=2b201cb79ba1478eaccd581214499848

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=kernel&flavor=default&distros=centos%2F7%2Fx86_64&sha1=a
    5269762c4b7bb3655b3e41168328fce85946603

[5199911]  multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
-----------------------------------------------------------------
time:   02:04:40
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199911/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199911/

    '/home/ubuntu/cephtest/archive/syslog/misc.log:2020-07-05T13:51:24.274119+0
    0:00 smithi189 crond[42958]: (root) INFO (Job execution of per-minute job
    scheduled for 13:50 delayed into subsequent minute 13:51. Skipping job
    run.) ' in syslog

[5199913]  multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199913/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199913/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=1b0a28c7a14543de970749a498eb2eaf

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=kernel&flavor=default&distros=centos%2F7%2Fx86_64&sha1=a
    5269762c4b7bb3655b3e41168328fce85946603

[5199915]  multimds/thrash/{0-supported-random-distro$/{centos_latest} begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199915/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199915/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=6ee790f7766a4c19bb6f25425eda0813

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=kernel&flavor=default&distros=centos%2F7%2Fx86_64&sha1=a
    5269762c4b7bb3655b3e41168328fce85946603

[5199917]  multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199917/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199917/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=c16e63c6946e4b61bc67afe0644a3620

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=kernel&flavor=default&distros=centos%2F7%2Fx86_64&sha1=a
    5269762c4b7bb3655b3e41168328fce85946603

[5199919]  multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/random/{k-testing supported$/{ubuntu_latest}} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
-----------------------------------------------------------------
time:   00:40:32
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199919/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199919/

    '/home/ubuntu/cephtest/archive/syslog/kern.log:2020-07-05T13:48:27.359403+0
    0:00 smithi132 kernel: [ 1437.528260] watchdog: BUG: soft lockup - CPU#3
    stuck for 22s! [kworker/3:1:28678] ' in syslog

[5199921]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{kernel-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   02:48:50
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199921/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199921/

    "2020-07-05 15:03:08.640081 mon.b (mon.0) 759 : cluster [WRN] Health check
    failed: 1 clients failing to respond to capability release
    (MDS_CLIENT_LATE_RELEASE)" in cluster log

[5199927]  multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199927/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199927/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=0f1d88c67cd34f1a8769403d3b325a09

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=kernel&flavor=default&distros=centos%2F7%2Fx86_64&sha1=a
    5269762c4b7bb3655b3e41168328fce85946603

[5199929]  multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199929/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199929/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=b41819c0cad34c4a810312663c597448

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=kernel&flavor=default&distros=centos%2F7%2Fx86_64&sha1=a
    5269762c4b7bb3655b3e41168328fce85946603

[5199931]  multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   00:26:42
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199931/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199931/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=54156d505e4e4a5685cc731210c267ec

    Command failed (workunit test suites/fsstress.sh) on smithi196 with status
    2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=120f482407ab4070126cb7ffc2f4f1b37517b4a0
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'

[5199941]  multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199941/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199941/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=e7ecf0f0b6354fcfaeaf371e0bd4b28f

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=kernel&flavor=default&distros=centos%2F7%2Fx86_64&sha1=a
    5269762c4b7bb3655b3e41168328fce85946603

[5199943]  multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199943/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199943/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=dbd6a49e47a849f69af380e037e57101

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=kernel&flavor=default&distros=centos%2F7%2Fx86_64&sha1=a
    5269762c4b7bb3655b3e41168328fce85946603



Pass
=================================================================
[5199900] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/random/{k-testing supported$/{ubuntu_16.04}} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:27:37
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199900/

[5199901] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:25:22
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199901/

[5199905] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{kernel-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
time:   02:28:55
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199905/

[5199908] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:21:48
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199908/

[5199909] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/random/{k-testing supported$/{ubuntu_latest}} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:12:33
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199909/

[5199923] multimds/thrash/{0-supported-random-distro$/{rhel_latest} begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/random/{k-testing supported$/{rhel_latest}} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:21:12
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199923/

[5199925] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:21:03
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199925/

[5199933] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/random/{k-testing supported$/{ubuntu_latest}} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:22:23
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199933/

[5199935] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{kernel-testing mount ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
time:   02:24:27
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199935/

[5199937] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   00:48:39
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199937/

[5199939] multimds/thrash/{0-supported-random-distro$/{ubuntu_16.04} begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:19:01
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199939/

[5199945] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/random/{k-testing supported$/{centos_latest}} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:22:37
info:   http://pulpito.ceph.com/teuthology-2020-07-05_13:15:02-multimds-nautilus-testing-basic-smithi/5199945/