2020-11-07T19:19:38.489 INFO:root:teuthology version: 1.0.0-41b569f
2020-11-07T19:19:38.492 INFO:teuthology.results:subset: '0/4'
2020-11-07T19:19:38.492 INFO:teuthology.results:seed: '9664'
2020-11-07T22:25:05.360 INFO:root:teuthology version: 1.0.0-41b569f
2020-11-07T22:25:05.398 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-11-08T02:44:13.546 INFO:teuthology.results:Tests finished! gathering results...
2020-11-08T02:44:59.501 INFO:scrape:Found 270 jobs
2020-11-08T02:45:20.540 INFO:scrape:Found 8 distinct failure reasons
2020-11-08T02:45:20.541 INFO:scrape:Failure: Command failed (workunit test fs/misc/dirfrag.sh) on smithi197 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9a5bca65a6339f282711b517ad3b4f7527967317 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/misc/dirfrag.sh'
2020-11-08T02:45:20.542 INFO:scrape:1 jobs: ['5599890']
2020-11-08T02:45:20.542 INFO:scrape:suites: ['begin', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'objectstore-ec/bluestore-comp', 'osd-asserts', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/stock/{k-stock', 'q_check_counter/check_counter', 'rhel_8}', 'session_timeout', 'tasks/cfuse_workunit_misc}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-11-08T02:45:20.542 INFO:scrape:
2020-11-08T02:45:20.543 INFO:scrape:Failure: "2020-11-07T20:22:53.084868+0000 mds.b (mds.0) 4 : cluster [WRN] Scrub error on inode 0x10000000002 (/d1/d2/d3) see mds.b log and `damage ls` output for details" in cluster log
2020-11-08T02:45:20.543 INFO:scrape:10 jobs: ['5599830', '5599938', '5599776', '5600019', '5599803', '5599992', '5599965', '5599911', '5599857', '5599884']
2020-11-08T02:45:20.544 INFO:scrape:suites intersection: ['begin', 'conf/{client', 'fuse-default-perm-no}', 'mds', 'mon', 'osd-asserts', 'osd}', 'overrides/{basic/{frag_enable', 'q_check_counter/check_counter', 'session_timeout', 'tasks/cephfs_test_multimds_misc}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-11-08T02:45:20.544 INFO:scrape:suites union: ['begin', 'clusters/3-mds', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'k-testing}', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'multimds/basic/{0-supported-random-distro$/{rhel_8}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'osd-asserts', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'q_check_counter/check_counter', 'rhel_8}', 'session_timeout', 'tasks/cephfs_test_multimds_misc}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-11-08T02:45:20.544 INFO:scrape:
2020-11-08T02:45:20.545 INFO:scrape:Crash: Command failed (workunit test suites/fsstress.sh) on smithi106 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9a5bca65a6339f282711b517ad3b4f7527967317 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'
2020-11-08T02:45:20.545 INFO:scrape:ceph version 16.0.0-7004-g5877d1ba (5877d1ba6b84da54e7f1f6615a1f312d19cebfd7) pacific (dev)
 1: /lib64/libpthread.so.0(+0x12dc0) [0xf9a1dc0]
 2: gsignal()
 3: abort()
 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x50b544d]
 5: /usr/lib64/ceph/libceph-common.so.2(+0x277616) [0x50b5616]
 6: (Monitor::~Monitor()+0xbd6) [0x3e75c6]
 7: (Monitor::~Monitor()+0xd) [0x3e764d]
 8: main()
 9: __libc_start_main()
 10: _start()
2020-11-08T02:45:20.545 INFO:scrape:1 jobs: ['5599786']
2020-11-08T02:45:20.545 INFO:scrape:suites: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{fuse-default-perm-no', 'session_timeout', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-11-08T02:45:20.546 INFO:scrape:
2020-11-08T02:45:20.546 INFO:scrape:Failure: 
2020-11-08T02:45:20.546 INFO:scrape:2 jobs: ['5599813', '5599867']
2020-11-08T02:45:20.547 INFO:scrape:suites intersection: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'multimds/verify/{begin', 'osd}', 'overrides/{fuse-default-perm-no', 'session_timeout', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-11-08T02:45:20.547 INFO:scrape:suites union: ['centos_latest', 'clusters/9-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{fuse-default-perm-no', 'session_timeout', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
2020-11-08T02:45:20.547 INFO:scrape:
2020-11-08T02:45:20.548 INFO:scrape:Failure: Command failed (workunit test suites/ffsb.sh) on smithi084 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9a5bca65a6339f282711b517ad3b4f7527967317 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'
2020-11-08T02:45:20.548 INFO:scrape:1 jobs: ['5599791']
2020-11-08T02:45:20.548 INFO:scrape:suites: ['begin', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest}', 'objectstore-ec/filestore-xfs', 'osd-asserts', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/stock/{k-stock', 'q_check_counter/check_counter', 'rhel_8}', 'session_timeout', 'tasks/cfuse_workunit_suites_ffsb}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-11-08T02:45:20.548 INFO:scrape:
2020-11-08T02:45:20.549 INFO:scrape:Failure: Test failure: test_ephemeral_pin_distribution (tasks.cephfs.test_exports.TestEphemeralPins)
2020-11-08T02:45:20.550 INFO:scrape:1 jobs: ['5599825']
2020-11-08T02:45:20.550 INFO:scrape:suites: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'objectstore-ec/bluestore-comp-ec-root', 'osd-asserts', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/testing/{flavor/centos_latest', 'q_check_counter/check_counter', 'session_timeout', 'tasks/cephfs_test_exports}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-11-08T02:45:20.551 INFO:scrape:
2020-11-08T02:45:20.551 INFO:scrape:Timeout 3h running clone.client.1/qa/workunits/fs/norstats/kernel_untar_tar.sh
2020-11-08T02:45:20.551 INFO:scrape:2 jobs: ['5600000', '5599784']
2020-11-08T02:45:20.552 INFO:scrape:suites intersection: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'osd-asserts', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/testing/{flavor/centos_latest', 'q_check_counter/check_counter', 'session_timeout', 'tasks/cfuse_workunit_norstats}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-11-08T02:45:20.552 INFO:scrape:suites union: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'osd-asserts', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/testing/{flavor/centos_latest', 'q_check_counter/check_counter', 'session_timeout', 'tasks/cfuse_workunit_norstats}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-11-08T02:45:20.552 INFO:scrape:
2020-11-08T02:45:20.553 INFO:scrape:Failure: Test failure: test_ephemeral_random_max (tasks.cephfs.test_exports.TestEphemeralPins)
2020-11-08T02:45:20.553 INFO:scrape:1 jobs: ['5599987']
2020-11-08T02:45:20.553 INFO:scrape:suites: ['begin', 'clusters/3-mds', 'conf/{client', 'fuse-default-perm-no}', 'mds', 'mon', 'mount/fuse', 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest}', 'objectstore-ec/filestore-xfs', 'osd-asserts', 'osd}', 'overrides/{basic/{frag_enable', 'q_check_counter/check_counter', 'session_timeout', 'tasks/cephfs_test_exports}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2020-11-08T02:45:20.553 INFO:scrape:
2020-11-08T02:45:20.568 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/
failed:  19
dead:    0
running: 0
waiting: 0
queued:  0
passed:  251


Fail
=================================================================
[5599776]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}
-----------------------------------------------------------------
time:   00:15:50
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599776/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599776/

    "2020-11-07T19:54:49.276805+0000 mds.b (mds.0) 4 : cluster [WRN] Scrub
    error on inode 0x10000000002 (/d1/d2/d3) see mds.b log and `damage ls`
    output for details" in cluster log

[5599784]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
-----------------------------------------------------------------
time:   03:13:34
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599784/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599784/
sentry: https://sentry.ceph.com/ceph/sepia/?query=e76beef1f3fc487296ad4f92c6fc6b15

    Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi049
    with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp &&
    cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9a5bca65a6339f282711b517ad3b4f7527967317
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.1/qa/workunits/fs/norstats/kernel_untar_tar.sh'

[5599786]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   00:45:39
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599786/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599786/
sentry: https://sentry.ceph.com/ceph/sepia/?query=405fd11965304111b95f1dc07667d736

    Command failed (workunit test suites/fsstress.sh) on smithi106 with status
    2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9a5bca65a6339f282711b517ad3b4f7527967317
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh'

[5599791]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
-----------------------------------------------------------------
time:   00:19:54
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599791/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599791/
sentry: https://sentry.ceph.com/ceph/sepia/?query=6ac4c2f8119f49649dd7c4fadf5c378b

    Command failed (workunit test suites/ffsb.sh) on smithi084 with status 1:
    'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9a5bca65a6339f282711b517ad3b4f7527967317
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

[5599803]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}
-----------------------------------------------------------------
time:   00:16:21
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599803/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599803/

    "2020-11-07T20:02:21.538936+0000 mds.c (mds.0) 4 : cluster [WRN] Scrub
    error on inode 0x10000000002 (/d1/d2/d3) see mds.c log and `damage ls`
    output for details" in cluster log

[5599813]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   03:49:08
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599813/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599813/
sentry: https://sentry.ceph.com/ceph/sepia/?query=debcecfe2bbc4df49adac2fe26c95541

[5599825]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:17:10
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599825/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599825/
sentry: https://sentry.ceph.com/ceph/sepia/?query=e5c70afa187d4bf1b3a7f947c9a5aa94

    Test failure: test_ephemeral_pin_distribution
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5599830]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}
-----------------------------------------------------------------
time:   00:13:52
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599830/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599830/

    "2020-11-07T20:22:53.084868+0000 mds.b (mds.0) 4 : cluster [WRN] Scrub
    error on inode 0x10000000002 (/d1/d2/d3) see mds.b log and `damage ls`
    output for details" in cluster log

[5599857]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}
-----------------------------------------------------------------
time:   00:17:56
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599857/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599857/

    "2020-11-07T20:48:59.308606+0000 mds.e (mds.0) 4 : cluster [WRN] Scrub
    error on inode 0x10000000002 (/d1/d2/d3) see mds.e log and `damage ls`
    output for details" in cluster log

[5599867]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   03:42:51
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599867/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599867/
sentry: https://sentry.ceph.com/ceph/sepia/?query=dfb952d6d67e41d4b517f7956acc7840

[5599884]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}
-----------------------------------------------------------------
time:   00:15:28
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599884/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599884/

    "2020-11-07T20:49:56.927546+0000 mds.a (mds.0) 4 : cluster [WRN] Scrub
    error on inode 0x10000000002 (/d1/d2/d3) see mds.a log and `damage ls`
    output for details" in cluster log

[5599890]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
-----------------------------------------------------------------
time:   00:14:23
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599890/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599890/
sentry: https://sentry.ceph.com/ceph/sepia/?query=689655b2edd6473ea1e97a837904fa45

    Command failed (workunit test fs/misc/dirfrag.sh) on smithi197 with status
    1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd --
    /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9a5bca65a6339f282711b517ad3b4f7527967317
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/misc/dirfrag.sh'

[5599911]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}
-----------------------------------------------------------------
time:   00:15:51
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599911/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599911/

    "2020-11-07T21:14:13.872641+0000 mds.a (mds.0) 4 : cluster [WRN] Scrub
    error on inode 0x10000000002 (/d1/d2/d3) see mds.a log and `damage ls`
    output for details" in cluster log

[5599938]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}
-----------------------------------------------------------------
time:   00:13:58
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599938/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599938/

    "2020-11-07T21:52:16.551451+0000 mds.b (mds.0) 4 : cluster [WRN] Scrub
    error on inode 0x10000000002 (/d1/d2/d3) see mds.b log and `damage ls`
    output for details" in cluster log

[5599965]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}
-----------------------------------------------------------------
time:   00:17:54
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599965/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599965/

    "2020-11-07T22:08:57.830574+0000 mds.c (mds.1) 2 : cluster [WRN] Scrub
    error on inode 0x10000000004 (/d1/d2/d3/d4/d5) see mds.c log and `damage
    ls` output for details" in cluster log

[5599987]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:45:19
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599987/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599987/
sentry: https://sentry.ceph.com/ceph/sepia/?query=eacaecac4bd24e2d8ca1e585e52b1566

    Test failure: test_ephemeral_random_max
    (tasks.cephfs.test_exports.TestEphemeralPins)

[5599992]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}
-----------------------------------------------------------------
time:   00:16:10
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599992/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599992/

    "2020-11-07T22:11:00.692302+0000 mds.c (mds.0) 4 : cluster [WRN] Scrub
    error on inode 0x10000000002 (/d1/d2/d3) see mds.c log and `damage ls`
    output for details" in cluster log

[5600000]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
-----------------------------------------------------------------
time:   03:12:33
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600000/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600000/
sentry: https://sentry.ceph.com/ceph/sepia/?query=d2e7a949dcd6428791ba7128a1a04610

    Command failed (workunit test fs/norstats/kernel_untar_tar.sh) on smithi180
    with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp &&
    cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9a5bca65a6339f282711b517ad3b4f7527967317
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.1/qa/workunits/fs/norstats/kernel_untar_tar.sh'

[5600019]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_multimds_misc}
-----------------------------------------------------------------
time:   00:15:07
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600019/
log:    http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600019/

    "2020-11-07T22:57:24.788426+0000 mds.b (mds.0) 4 : cluster [WRN] Scrub
    error on inode 0x10000000002 (/d1/d2/d3) see mds.b log and `damage ls`
    output for details" in cluster log



Pass
=================================================================
[5599771] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:48:46
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599771/

[5599772] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:12:34
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599772/

[5599773] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:26:47
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599773/

[5599774] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:20:07
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599774/

[5599775] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:54
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599775/

[5599777] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:16:00
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599777/

[5599778] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:18:11
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599778/

[5599779] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:15
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599779/

[5599780] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   02:13:14
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599780/

[5599781] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:13:50
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599781/

[5599782] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:11:53
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599782/

[5599783] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:28
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599783/

[5599785] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:15:20
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599785/

[5599787] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:54:43
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599787/

[5599788] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:38
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599788/

[5599789] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:43:51
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599789/

[5599790] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:49
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599790/

[5599792] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:17
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599792/

[5599793] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:15:59
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599793/

[5599794] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:33
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599794/

[5599795] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:57:33
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599795/

[5599796] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:42
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599796/

[5599797] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:12:50
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599797/

[5599798] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:42:43
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599798/

[5599799] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:13:02
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599799/

[5599800] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:43:49
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599800/

[5599801] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:17:38
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599801/

[5599802] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:54
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599802/

[5599804] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:13:51
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599804/

[5599805] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:24:28
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599805/

[5599806] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:24
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599806/

[5599807] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:08:31
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599807/

[5599808] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:31
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599808/

[5599809] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   02:00:13
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599809/

[5599810] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:20
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599810/

[5599811] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:12:32
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599811/

[5599812] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:17
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599812/

[5599814] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:26:00
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599814/

[5599815] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:32
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599815/

[5599816] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:26:07
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599816/

[5599817] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:16:28
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599817/

[5599818] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:46:04
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599818/

[5599819] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:44
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599819/

[5599820] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:19:36
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599820/

[5599821] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:32
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599821/

[5599822] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:32:11
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599822/

[5599823] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:28
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599823/

[5599824] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:13:28
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599824/

[5599826] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:13:54
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599826/

[5599827] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   00:57:23
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599827/

[5599828] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:17:35
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599828/

[5599829] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:48
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599829/

[5599831] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:44
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599831/

[5599832] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:19:14
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599832/

[5599833] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:40
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599833/

[5599834] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:23:31
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599834/

[5599835] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:15:04
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599835/

[5599836] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:37:44
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599836/

[5599837] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:34
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599837/

[5599838] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:15:38
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599838/

[5599839] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:14
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599839/

[5599840] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
time:   02:34:12
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599840/

[5599841] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:56:15
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599841/

[5599842] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:23
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599842/

[5599843] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   00:53:24
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599843/

[5599844] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:15:34
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599844/

[5599845] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:30:50
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599845/

[5599846] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:14:45
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599846/

[5599847] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:24:21
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599847/

[5599848] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:13:33
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599848/

[5599849] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:27:56
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599849/

[5599850] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:33
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599850/

[5599851] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:12:12
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599851/

[5599852] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:40:52
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599852/

[5599853] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:10
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599853/

[5599854] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:42:33
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599854/

[5599855] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:16:46
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599855/

[5599856] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:50
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599856/

[5599858] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:02
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599858/

[5599859] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:18:08
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599859/

[5599860] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:11:49
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599860/

[5599861] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:41:34
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599861/

[5599862] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:11:28
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599862/

[5599863] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   00:46:14
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599863/

[5599864] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:18
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599864/

[5599865] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:14:45
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599865/

[5599866] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:03
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599866/

[5599868] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:54:36
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599868/

[5599869] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:46
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599869/

[5599870] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:11:57
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599870/

[5599871] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:57
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599871/

[5599872] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   01:50:35
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599872/

[5599873] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:22
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599873/

[5599874] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:17:15
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599874/

[5599875] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:05
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599875/

[5599876] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:24:15
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599876/

[5599877] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:13
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599877/

[5599878] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:11:54
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599878/

[5599879] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:53:22
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599879/

[5599880] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:30
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599880/

[5599881] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep}
time:   01:40:06
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599881/

[5599882] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:20:03
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599882/

[5599883] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:14:03
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599883/

[5599885] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:01
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599885/

[5599886] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:17:56
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599886/

[5599887] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:28
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599887/

[5599888] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   03:39:42
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599888/

[5599889] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:12:34
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599889/

[5599891] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:06
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599891/

[5599892] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:15:24
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599892/

[5599893] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:13:10
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599893/

[5599894] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
time:   01:59:57
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599894/

[5599895] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:30:27
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599895/

[5599896] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:52
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599896/

[5599897] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   02:02:24
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599897/

[5599898] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:16:11
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599898/

[5599899] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   01:54:05
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599899/

[5599900] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:02
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599900/

[5599901] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:15:51
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599901/

[5599902] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:15:41
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599902/

[5599903] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:43:10
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599903/

[5599904] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:11:50
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599904/

[5599905] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:09:13
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599905/

[5599906] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:42:07
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599906/

[5599907] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:43
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599907/

[5599908] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:14:19
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599908/

[5599909] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:18:42
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599909/

[5599910] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:21:52
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599910/

[5599912] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:31
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599912/

[5599913] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:24:01
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599913/

[5599914] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:14
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599914/

[5599915] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   00:37:17
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599915/

[5599916] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:43
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599916/

[5599917] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:33:16
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599917/

[5599918] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:22:12
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599918/

[5599919] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:15:41
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599919/

[5599920] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:26
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599920/

[5599921] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   00:57:06
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599921/

[5599922] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:25:15
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599922/

[5599923] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:49
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599923/

[5599924] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:35:04
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599924/

[5599925] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:11:07
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599925/

[5599926] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:29:03
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599926/

[5599927] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:17
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599927/

[5599928] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:17:05
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599928/

[5599929] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:38
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599929/

[5599930] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   01:14:44
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599930/

[5599931] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:23:55
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599931/

[5599932] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:13:11
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599932/

[5599933] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:40:51
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599933/

[5599934] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:33
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599934/

[5599935] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:14:28
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599935/

[5599936] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:17:16
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599936/

[5599937] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:14
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599937/

[5599939] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:35
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599939/

[5599940] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:19:53
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599940/

[5599941] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:19:37
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599941/

[5599942] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:25:35
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599942/

[5599943] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:11:00
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599943/

[5599944] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:37:55
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599944/

[5599945] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:23:07
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599945/

[5599946] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:19:27
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599946/

[5599947] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:23
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599947/

[5599948] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   00:56:27
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599948/

[5599949] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:48:09
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599949/

[5599950] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:23
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599950/

[5599951] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:29:57
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599951/

[5599952] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:04
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599952/

[5599953] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:38:31
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599953/

[5599954] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:24:52
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599954/

[5599955] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:17:15
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599955/

[5599956] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:13
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599956/

[5599957] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:20:00
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599957/

[5599958] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:22:39
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599958/

[5599959] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:11:58
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599959/

[5599960] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:41:52
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599960/

[5599961] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:26
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599961/

[5599962] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:14:12
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599962/

[5599963] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:16:46
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599963/

[5599964] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:19:52
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599964/

[5599966] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:29
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599966/

[5599967] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:17:45
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599967/

[5599968] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:19:08
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599968/

[5599969] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:04:51
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599969/

[5599970] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:14:11
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599970/

[5599971] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   02:29:49
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599971/

[5599972] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:22:33
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599972/

[5599973] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:15:12
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599973/

[5599974] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:18
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599974/

[5599975] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   00:54:21
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599975/

[5599976] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:46:00
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599976/

[5599977] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:23
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599977/

[5599978] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   00:50:21
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599978/

[5599979] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:40
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599979/

[5599980] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:36:51
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599980/

[5599981] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:21:42
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599981/

[5599982] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:24:02
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599982/

[5599983] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:51
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599983/

[5599984] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:27:28
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599984/

[5599985] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:17
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599985/

[5599986] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:39:56
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599986/

[5599988] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:36
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599988/

[5599989] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:21:10
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599989/

[5599990] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:20:36
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599990/

[5599991] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:47
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599991/

[5599993] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:38
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599993/

[5599994] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:18:01
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599994/

[5599995] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:19:02
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599995/

[5599996] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   03:35:11
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599996/

[5599997] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:10:07
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599997/

[5599998] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   00:48:17
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599998/

[5599999] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:25:18
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5599999/

[5600001] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:11:45
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600001/

[5600002] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   00:58:27
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600002/

[5600003] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:48:29
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600003/

[5600004] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:54
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600004/

[5600005] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:06:52
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600005/

[5600006] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:09:54
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600006/

[5600007] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   02:32:24
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600007/

[5600008] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:21:23
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600008/

[5600009] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:14:56
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600009/

[5600010] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:13:04
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600010/

[5600011] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:25:46
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600011/

[5600012] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:41
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600012/

[5600013] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:10:26
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600013/

[5600014] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:42:17
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600014/

[5600015] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:09:25
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600015/

[5600016] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:17:33
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600016/

[5600017] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_mds_metrics}
time:   00:18:57
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600017/

[5600018] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:47
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600018/

[5600020] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:25
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600020/

[5600021] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:26:43
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600021/

[5600022] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:23:25
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600022/

[5600023] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   01:48:08
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600023/

[5600024] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:10:55
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600024/

[5600025] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:23:23
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600025/

[5600026] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:23:25
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600026/

[5600027] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:15:15
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600027/

[5600028] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:02
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600028/

[5600029] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   01:00:21
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600029/

[5600030] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:20:16
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600030/

[5600031] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:55
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600031/

[5600032] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   01:30:25
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600032/

[5600033] multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/osd clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:09:46
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600033/

[5600034] multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   02:33:53
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600034/

[5600035] multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:21:34
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600035/

[5600036] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:16:14
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600036/

[5600037] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mon clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:12:36
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600037/

[5600038] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   01:02:48
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600038/

[5600039] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/osd clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:53
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600039/

[5600040] multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/9-mds conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:12:20
info:   http://pulpito.ceph.com/pdonnell-2020-11-07_19:18:17-multimds-wip-pdonnell-testing-20201107.025143-distro-basic-smithi/5600040/