2021-05-20T21:58:35.387 INFO:root:teuthology version: 1.1.0-471a2f1e
2021-05-20T21:58:35.389 INFO:teuthology.results:subset: '1455/2999'
2021-05-20T21:58:35.389 INFO:teuthology.results:seed: '54'
2021-05-21T07:54:37.266 INFO:root:teuthology version: 1.1.0-471a2f1e
2021-05-21T07:54:37.313 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-05-21T08:24:39.430 INFO:teuthology.results:Tests finished! gathering results...
2021-05-21T08:24:43.994 INFO:scrape:Found 49 jobs
2021-05-21T08:24:45.276 INFO:scrape:Found 3 distinct failure reasons
2021-05-21T08:24:45.277 INFO:scrape:Timeout 120 running ceph --cluster ceph -m 172.21.2.137 mon_status
2021-05-21T08:24:45.277 INFO:scrape:1 jobs: ['6115373']
2021-05-21T08:24:45.278 INFO:scrape:suites: ['clusters/1-mds-1-client', 'conf/{client', 'kcephfs/thrash/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp-ec-root', 'osd-asserts', 'osd}', 'overrides/{distro/rhel/{k-distro', 'overrides/{frag_enable', 'rhel_latest}', 'thrash-health-whitelist', 'thrashers/mon', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workloads/kclient_workunit_suites_ffsb}']
2021-05-21T08:24:45.278 INFO:scrape:
2021-05-21T08:24:45.279 INFO:scrape:Failure: Test failure: test_evicted_caps (tasks.cephfs.test_client_recovery.TestClientRecovery)
2021-05-21T08:24:45.279 INFO:scrape:1 jobs: ['6115354']
2021-05-21T08:24:45.280 INFO:scrape:suites: ['clusters/1-mds-4-client', 'conf/{client', 'kcephfs/recovery/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp', 'osd-asserts', 'osd}', 'overrides/{distro/rhel/{k-distro', 'overrides/{frag_enable', 'rhel_latest}', 'tasks/client-recovery}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2021-05-21T08:24:45.280 INFO:scrape:
2021-05-21T08:24:45.280 INFO:scrape:Failure: Command failed (workunit test suites/ffsb.sh) on gibba031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5ef401921d7a88aea18ec7558f7f9374ebd8f5a6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'
2021-05-21T08:24:45.281 INFO:scrape:1 jobs: ['6115340']
2021-05-21T08:24:45.281 INFO:scrape:suites: ['clusters/1-mds-1-client', 'conf/{client', 'kcephfs/thrash/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp', 'osd-asserts', 'osd}', 'overrides/{distro/rhel/{k-distro', 'overrides/{frag_enable', 'rhel_latest}', 'thrash-health-whitelist', 'thrashers/mds', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workloads/kclient_workunit_suites_ffsb}']
2021-05-21T08:24:45.282 INFO:scrape:
2021-05-21T08:24:45.284 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba
=================================================================
info:    http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/
failed:  3
dead:    0
running: 0
waiting: 0
queued:  0
passed:  46


Fail
=================================================================
[6115340]  kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/mds workloads/kclient_workunit_suites_ffsb}
-----------------------------------------------------------------
time:   00:35:38
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115340/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115340/
sentry: https://sentry.ceph.com/organizations/ceph/?query=96a2840fe6764e94bc9b38331d893630

    Command failed (workunit test suites/ffsb.sh) on gibba031 with status 1:
    'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=5ef401921d7a88aea18ec7558f7f9374ebd8f5a6
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh'

[6115354]  kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery}
-----------------------------------------------------------------
time:   00:38:40
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115354/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115354/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a45b97cd87c74f648a1159ac1c33e11b

    Test failure: test_evicted_caps
    (tasks.cephfs.test_client_recovery.TestClientRecovery)

[6115373]  kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/mon workloads/kclient_workunit_suites_ffsb}
-----------------------------------------------------------------
time:   00:54:44
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115373/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115373/

    Command failed on gibba011 with status 124: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph -m 172.21.2.137 mon_status'



Pass
=================================================================
[6115330] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush}
time:   00:18:31
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115330/

[6115331] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:49:16
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115331/

[6115332] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/default workloads/kclient_workunit_suites_iozone}
time:   00:19:03
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115332/

[6115333] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsync}
time:   00:14:59
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115333/

[6115334] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/mds-full}
time:   00:39:35
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115334/

[6115335] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:32:04
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115335/

[6115336] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm}
time:   00:19:43
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115336/

[6115337] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_iozone}
time:   00:18:07
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115337/

[6115338] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap}
time:   00:23:39
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115338/

[6115339] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_pjd}
time:   00:15:32
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115339/

[6115341] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:47:16
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115341/

[6115342] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/strays}
time:   00:37:21
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115342/

[6115343] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_trivial_sync}
time:   00:14:04
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115343/

[6115344] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/volume-client}
time:   00:51:14
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115344/

[6115345] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:31:16
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115345/

[6115346] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair}
time:   00:19:32
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115346/

[6115347] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_direct_io}
time:   00:14:24
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115347/

[6115348] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/mon workloads/kclient_workunit_suites_iozone}
time:   00:16:54
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115348/

[6115349] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/backtrace}
time:   00:18:30
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115349/

[6115350] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_kernel_untar_build}
time:   00:23:54
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115350/

[6115351] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:44:32
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115351/

[6115352] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/client-limits}
time:   00:29:18
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115352/

[6115353] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_misc}
time:   00:29:59
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115353/

[6115355] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:26:47
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115355/

[6115356] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/default workloads/kclient_workunit_suites_ffsb}
time:   00:28:13
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115356/

[6115357] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_o_trunc}
time:   00:24:01
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115357/

[6115358] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/damage}
time:   00:32:58
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115358/

[6115359] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/data-scan}
time:   00:39:26
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115359/

[6115360] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_snaps}
time:   00:43:36
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115360/

[6115361] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:58:14
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115361/

[6115362] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/failover}
time:   00:36:26
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115362/

[6115363] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_dbench}
time:   00:45:51
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115363/

[6115364] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/mds workloads/kclient_workunit_suites_iozone}
time:   00:16:32
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115364/

[6115365] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub}
time:   00:22:35
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115365/

[6115366] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:27:31
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115366/

[6115367] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_ffsb}
time:   00:34:25
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115367/

[6115368] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair}
time:   00:28:48
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115368/

[6115369] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush}
time:   00:18:43
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115369/

[6115370] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:56:59
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115370/

[6115371] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsstress}
time:   00:17:35
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115371/

[6115372] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/mds-full}
time:   00:39:14
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115372/

[6115374] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsx}
time:   00:18:42
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115374/

[6115375] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:25:39
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115375/

[6115376] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm}
time:   00:19:35
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115376/

[6115377] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsync}
time:   00:14:59
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115377/

[6115378] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap}
time:   00:23:17
info:   http://pulpito.ceph.com/teuthology-2021-05-14_13:25:02-kcephfs-nautilus-distro-basic-gibba/6115378/