2022-03-04T03:10:41.623 INFO:root:teuthology version: 1.1.0-eeaeee38
2022-03-04T03:10:41.625 INFO:teuthology.results:subset: '8769/9999'
2022-03-04T03:10:41.625 INFO:teuthology.results:seed: '9397'
2022-03-04T03:29:34.141 INFO:root:teuthology version: 1.1.0-eeaeee38
2022-03-04T03:29:34.178 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-03-04T03:44:35.311 INFO:teuthology.results:Tests finished! gathering results...
2022-03-04T03:44:36.626 INFO:scrape:Found 24 jobs
2022-03-04T03:44:37.681 INFO:scrape:Found 1 distinct failure reasons
2022-03-04T03:44:37.682 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7516df7fec9d21581a28fd4addedfd8e4c26f0d9
2022-03-04T03:44:37.683 INFO:scrape:24 jobs: ['6708108', '6708121', '6708119', '6708099', '6708112', '6708104', '6708110', '6708122', '6708116', '6708120', '6708113', '6708115', '6708101', '6708106', '6708118', '6708105', '6708102', '6708107', '6708109', '6708103', '6708117', '6708100', '6708114', '6708111']
2022-03-04T03:44:37.683 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'session_timeout', 'whitelist_health']
2022-03-04T03:44:37.684 INFO:scrape:suites union: ['begin', 'centos_latest', 'ceph-thrash/mds', 'ceph-thrash/mon', 'clusters/3-mds', 'clusters/3-mds-2-standby', 'clusters/9-mds', 'clusters/9-mds-3-standby', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'inline/yes', 'k-testing}', 'mds', 'mon', 'mon-debug', 'mount', 'mount/fuse', 'mount/kclient/{k-testing', 'mount/kclient/{mount', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'multimds/basic/{0-supported-random-distro$/{rhel_8}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest}', 'multimds/thrash/{0-supported-random-distro$/{centos_8}', 'multimds/thrash/{0-supported-random-distro$/{rhel_8}', 'multimds/thrash/{0-supported-random-distro$/{ubuntu_18.04}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{fuse-default-perm-no', 'q_check_counter/check_counter', 'rhel_8}', 'session_timeout', 'tasks/cephfs_test_exports}', 'tasks/cephfs_test_snapshots}', 'tasks/cfuse_workunit_kernel_untar_build}', 'tasks/cfuse_workunit_misc}', 'tasks/cfuse_workunit_norstats}', 'tasks/cfuse_workunit_suites_blogbench}', 'tasks/cfuse_workunit_suites_dbench', 'tasks/cfuse_workunit_suites_dbench}', 'tasks/cfuse_workunit_suites_ffsb}', 'tasks/cfuse_workunit_suites_fsstress', 'tasks/cfuse_workunit_suites_fsstress}', 'tasks/cfuse_workunit_suites_fsx}', 'tasks/cfuse_workunit_suites_pjd}', 'thrash/{frag_enable', 'thrash_debug}', 'validater/lockdep}', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'whitelist_wrongly_marked_down}}']
2022-03-04T03:44:37.684 INFO:scrape:
2022-03-04T03:44:37.686 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/
failed:  24
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[6708099]  multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   00:04:56
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708099/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708099/
sentry: https://sentry.ceph.com/organizations/ceph/?query=81ab52fdc0ec4c56861f455725781b6c

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708100]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
-----------------------------------------------------------------
time:   00:06:22
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708100/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708100/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3fc5422df55741a2ba790f863ffe72bc

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708101]  multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   00:06:10
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708101/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708101/
sentry: https://sentry.ceph.com/organizations/ceph/?query=00d46509807d4b69a1176cbfc1de8472

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708102]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
-----------------------------------------------------------------
time:   00:06:11
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708102/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708102/
sentry: https://sentry.ceph.com/organizations/ceph/?query=311f9261fd694176add0947bf78b654d

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708103]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
-----------------------------------------------------------------
time:   00:06:12
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708103/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708103/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0f23031dd4be4d699dac7bb098d60167

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708104]  multimds/basic/{0-supported-random-distro$/{rhel_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
-----------------------------------------------------------------
time:   00:04:22
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708104/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708104/
sentry: https://sentry.ceph.com/organizations/ceph/?query=231418e9b1e4431fa54eb10261fbe067

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1
    =7516df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708105]  multimds/thrash/{0-supported-random-distro$/{rhel_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
-----------------------------------------------------------------
time:   00:04:24
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708105/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708105/
sentry: https://sentry.ceph.com/organizations/ceph/?query=22a68bc9acc04cd88c64f71ee28462d8

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1
    =7516df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708106]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
-----------------------------------------------------------------
time:   00:05:02
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708106/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708106/
sentry: https://sentry.ceph.com/organizations/ceph/?query=db295a6f97ef42a2a44ed74a70ffa4f6

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708107]  multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
-----------------------------------------------------------------
time:   00:04:48
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708107/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708107/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d3dbb1695078410382623d1be160c31c

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708108]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
-----------------------------------------------------------------
time:   00:06:40
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708108/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708108/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0ec9ac3f194b4126a88ecaf81023f9e6

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708109]  multimds/thrash/{0-supported-random-distro$/{ubuntu_18.04} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
-----------------------------------------------------------------
time:   00:03:35
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708109/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708109/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c0acc0af321c4e04a3dd8ebd046e4072

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1
    =7516df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708110]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   00:06:14
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708110/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708110/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b5fb35cd602c494495996a64710daae1

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708111]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
-----------------------------------------------------------------
time:   00:06:12
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708111/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708111/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b0678ed3f8e142ca9a231bcf554333b6

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708112]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
-----------------------------------------------------------------
time:   00:04:21
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708112/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708112/
sentry: https://sentry.ceph.com/organizations/ceph/?query=87ffec7dc33248af9b7e9751211c35e5

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1
    =7516df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708113]  multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   00:06:22
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708113/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708113/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3f942b13fe364a3d876e36f38501e6f2

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708114]  multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
-----------------------------------------------------------------
time:   00:03:22
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708114/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708114/
sentry: https://sentry.ceph.com/organizations/ceph/?query=48b503ad8ca749808bb47a3ab91d56b3

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1
    =7516df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708115]  multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
-----------------------------------------------------------------
time:   00:04:55
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708115/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708115/
sentry: https://sentry.ceph.com/organizations/ceph/?query=555a469e73d8434ea5c1f43696fa0ef8

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708116]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
-----------------------------------------------------------------
time:   00:06:43
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708116/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708116/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7c64a00c0c344c99bef6a6424565cd58

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708117]  multimds/thrash/{0-supported-random-distro$/{ubuntu_18.04} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
-----------------------------------------------------------------
time:   00:06:13
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708117/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708117/
sentry: https://sentry.ceph.com/organizations/ceph/?query=69ebd55eb7274a0bb042d631ea2a7752

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708118]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
-----------------------------------------------------------------
time:   00:06:14
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708118/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708118/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ec18f949c2e744cea89bad64b77e49cd

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708119]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
-----------------------------------------------------------------
time:   00:06:14
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708119/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708119/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0c7754f433e64edbb76f0b9e6a2e5243

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708120]  multimds/basic/{0-supported-random-distro$/{centos_8} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
-----------------------------------------------------------------
time:   00:04:25
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708120/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708120/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1c5ac1cda871427bbb72cb9ceb88d203

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1
    =7516df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708121]  multimds/thrash/{0-supported-random-distro$/{centos_8} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable session_timeout whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   00:04:26
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708121/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708121/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1651c141bad84f2e88049f8212b6cba1

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1
    =7516df7fec9d21581a28fd4addedfd8e4c26f0d9

[6708122]  multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
-----------------------------------------------------------------
time:   00:04:54
info:   http://pulpito.ceph.com/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708122/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-26_13:15:03-multimds-octopus-distro-default-smithi/6708122/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f2174a58d00d46218649f92bda19f68f

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=751
    6df7fec9d21581a28fd4addedfd8e4c26f0d9