2022-07-20T22:01:00.293 INFO:root:teuthology version: 1.1.1.dev383+g89d874d5
2022-07-20T22:01:00.294 INFO:teuthology.results:subset: '111/120000'
2022-07-20T22:01:00.295 INFO:teuthology.results:seed: '3032'
2022-07-20T23:18:00.759 INFO:root:teuthology version: 1.1.1.dev383+g89d874d5
2022-07-20T23:18:00.805 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-07-21T00:07:05.089 INFO:teuthology.results:Tests finished! gathering results...
2022-07-21T00:07:11.758 INFO:scrape:Found 43 jobs
2022-07-21T00:07:16.732 INFO:scrape:Found 8 distinct failure reasons
2022-07-21T00:07:16.733 INFO:scrape:Failure: {'smithi075.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi139.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi202.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}}
2022-07-21T00:07:16.734 INFO:scrape:2 jobs: ['6940819', '6940829']
2022-07-21T00:07:16.735 INFO:scrape:suites intersection: ['conf/{client', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-07-21T00:07:16.736 INFO:scrape:suites union: ['clusters/1-mds-1-client', 'clusters/1-mds-4-client', 'conf/{client', 'inline/yes', 'kcephfs/cephfs/{begin', 'kcephfs/recovery/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp', 'objectstore-ec/filestore-xfs', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/kclient_workunit_suites_fsstress}', 'tasks/pool-perm}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-07-21T00:07:16.736 INFO:scrape:
2022-07-21T00:07:16.737 INFO:scrape:Failure: {'smithi115.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi117.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi203.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi012.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi120.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}, 'smithi204.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}}
2022-07-21T00:07:16.737 INFO:scrape:1 jobs: ['6940818']
2022-07-21T00:07:16.738 INFO:scrape:suites: ['clusters/1-mds-4-client', 'conf/{client', 'kcephfs/recovery/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/filestore-xfs', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/journal-repair}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-07-21T00:07:16.739 INFO:scrape:
2022-07-21T00:07:16.739 INFO:scrape:Failure: Test failure: test_acls (tasks.cephfs.test_acls.TestACLs)
2022-07-21T00:07:16.740 INFO:scrape:1 jobs: ['6940796']
2022-07-21T00:07:16.741 INFO:scrape:suites: ['clusters/1-mds-1-client', 'conf/{client', 'inline/yes', 'k-testing}', 'kcephfs/cephfs/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp-ec-root', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/acls-kernel-client}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-07-21T00:07:16.741 INFO:scrape:
2022-07-21T00:07:16.742 INFO:scrape:Failure: "2022-07-20T23:00:11.616391+0000 osd.1 (osd.1) 21 : cluster [ERR] 4.4s0 deep-scrub : stat mismatch, got 0/1 objects, 0/0 clones, 0/1 dirty, 0/0 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 0/4194304 bytes, 0/0 manifest objects, 0/0 hit_set_archive bytes." in cluster log
2022-07-21T00:07:16.743 INFO:scrape:1 jobs: ['6940795']
2022-07-21T00:07:16.744 INFO:scrape:suites: ['clusters/1-mds-1-client', 'conf/{client', 'k-testing}', 'kcephfs/thrash/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp-ec-root', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'thrash-health-whitelist', 'thrashers/default', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workloads/kclient_workunit_suites_ffsb}']
2022-07-21T00:07:16.744 INFO:scrape:
2022-07-21T00:07:16.745 INFO:scrape:Failure: Cannot connect to remote host smithi055
2022-07-21T00:07:16.746 INFO:scrape:1 jobs: ['6940804']
2022-07-21T00:07:16.746 INFO:scrape:suites: ['clusters/1-mds-4-client', 'conf/{client', 'k-testing}', 'kcephfs/recovery/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/filestore-xfs', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/client-recovery}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-07-21T00:07:16.747 INFO:scrape:
2022-07-21T00:07:16.748 INFO:scrape:Failure: Command failed (workunit test suites/dbench.sh) on smithi096 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57544da114dbbe1c44f27b2ea18e3233e4e917e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh'
2022-07-21T00:07:16.749 INFO:scrape:1 jobs: ['6940813']
2022-07-21T00:07:16.749 INFO:scrape:suites: ['clusters/1-mds-1-client', 'conf/{client', 'inline/yes', 'k-testing}', 'kcephfs/cephfs/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/kclient_workunit_suites_dbench}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-07-21T00:07:16.750 INFO:scrape:
2022-07-21T00:07:16.751 INFO:scrape:Failure: Test failure: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays)
2022-07-21T00:07:16.751 INFO:scrape:1 jobs: ['6940791']
2022-07-21T00:07:16.752 INFO:scrape:suites: ['clusters/1-mds-4-client', 'conf/{client', 'k-testing}', 'kcephfs/recovery/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-bitmap', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/strays}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-07-21T00:07:16.753 INFO:scrape:
2022-07-21T00:07:16.754 INFO:scrape:Failure: Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull)
2022-07-21T00:07:16.754 INFO:scrape:1 jobs: ['6940827']
2022-07-21T00:07:16.755 INFO:scrape:suites: ['clusters/1-mds-4-client', 'conf/{client', 'k-testing}', 'kcephfs/recovery/{begin', 'kclient/{mount', 'log-config', 'mds', 'mon', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-comp-ec-root', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-07-21T00:07:16.756 INFO:scrape:
2022-07-21T00:07:16.760 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/
failed:  6
dead:    3
running: 0
waiting: 0
queued:  0
passed:  34


Fail
=================================================================
[6940791]  kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/strays}
-----------------------------------------------------------------
time:   00:15:35
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940791/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940791/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c42a4dee6f194b13adc2fa5f511f542b

    Test failure: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays)

[6940795]  kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/default workloads/kclient_workunit_suites_ffsb}
-----------------------------------------------------------------
time:   00:42:51
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940795/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940795/

    "2022-07-20T23:00:11.616391+0000 osd.1 (osd.1) 21 : cluster [ERR] 4.4s0
    deep-scrub : stat mismatch, got 0/1 objects, 0/0 clones, 0/1 dirty, 0/0
    omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 0/4194304 bytes, 0/0
    manifest objects, 0/0 hit_set_archive bytes." in cluster log

[6940796]  kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/acls-kernel-client}
-----------------------------------------------------------------
time:   00:14:54
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940796/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940796/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7dc3cd40b2214278bdbc730565e121dd

    Test failure: test_acls (tasks.cephfs.test_acls.TestACLs)

[6940804]  kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery}
-----------------------------------------------------------------
time:   00:49:05
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940804/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940804/

    Cannot connect to remote host smithi055

[6940813]  kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_dbench}
-----------------------------------------------------------------
time:   00:16:28
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940813/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940813/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6f113225b7d943d29ba0e556f2ab9385

    Command failed (workunit test suites/dbench.sh) on smithi096 with status 1:
    'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=57544da114dbbe1c44f27b2ea18e3233e4e917e0
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh'

[6940827]  kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/mds-full}
-----------------------------------------------------------------
time:   00:19:22
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940827/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940827/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8de45cc211a44036ac85f50e2bd80904

    Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull)



Dead
=================================================================
[6940818]  kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair}
-----------------------------------------------------------------
time:   00:22:43
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940818/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940818/
sentry: https://sentry.ceph.com/organizations/ceph/?query=345efdc32de54c339e0b3d54ae44e066

    {'smithi115.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output
    has been hidden due to the fact that 'no_log: true' was specified for this
    result", 'changed': True}, 'smithi117.front.sepia.ceph.com': {'attempts':
    12, 'censored': "the output has been hidden due to the fact that 'no_log:
    true' was specified for this result", 'changed': True},
    'smithi203.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output
    has been hidden due to the fact that 'no_log: true' was specified for this
    result", 'changed': True}, 'smithi012.front.sepia.ceph.com': {'attempts':
    12, 'censored': "the output has been hidden due to the fact that 'no_log:
    true' was specified for this result", 'changed': True},
    'smithi120.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output
    has been hidden due to the fact that 'no_log: true' was specified for this
    result", 'changed': True}, 'smithi204.front.sepia.ceph.com': {'attempts':
    12, 'censored': "the output has been hidden due to the fact that 'no_log:
    true' was specified for this result", 'changed': True}}

[6940819]  kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   00:15:11
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940819/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940819/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5550ff5b17b242e5b3ca20daa110b8f8

    {'smithi075.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output
    has been hidden due to the fact that 'no_log: true' was specified for this
    result", 'changed': True}, 'smithi139.front.sepia.ceph.com': {'attempts':
    12, 'censored': "the output has been hidden due to the fact that 'no_log:
    true' was specified for this result", 'changed': True},
    'smithi202.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output
    has been hidden due to the fact that 'no_log: true' was specified for this
    result", 'changed': True}}

[6940829]  kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm}
-----------------------------------------------------------------
time:   00:21:25
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940829/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940829/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0750ff54fb3f4bd797a79054dce46bb3

    {'smithi142.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output
    has been hidden due to the fact that 'no_log: true' was specified for this
    result", 'changed': True}, 'smithi143.front.sepia.ceph.com': {'attempts':
    12, 'censored': "the output has been hidden due to the fact that 'no_log:
    true' was specified for this result", 'changed': True},
    'smithi171.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output
    has been hidden due to the fact that 'no_log: true' was specified for this
    result", 'changed': True}, 'smithi081.front.sepia.ceph.com': {'attempts':
    12, 'censored': "the output has been hidden due to the fact that 'no_log:
    true' was specified for this result", 'changed': True},
    'smithi001.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output
    has been hidden due to the fact that 'no_log: true' was specified for this
    result", 'changed': True}, 'smithi161.front.sepia.ceph.com': {'attempts':
    12, 'censored': "the output has been hidden due to the fact that 'no_log:
    true' was specified for this result", 'changed': True}}



Pass
=================================================================
[6940784] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsync}
time:   00:13:13
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940784/

[6940785] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm}
time:   00:14:11
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940785/

[6940786] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:50:00
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940786/

[6940787] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/mon workloads/kclient_workunit_suites_iozone}
time:   00:13:36
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940787/

[6940788] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_iozone}
time:   00:18:50
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940788/

[6940789] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap}
time:   00:20:24
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940789/

[6940790] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_pjd}
time:   00:14:49
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940790/

[6940792] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_trivial_sync}
time:   00:14:12
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940792/

[6940793] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/volume-client}
time:   00:26:30
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940793/

[6940794] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:56:47
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940794/

[6940797] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair}
time:   00:18:24
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940797/

[6940798] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/backtrace}
time:   00:14:06
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940798/

[6940799] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_direct_io}
time:   00:14:10
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940799/

[6940800] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:38:35
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940800/

[6940801] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/mds workloads/kclient_workunit_suites_iozone}
time:   00:19:09
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940801/

[6940802] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/client-limits}
time:   00:19:42
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940802/

[6940803] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_kernel_untar_build}
time:   00:35:47
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940803/

[6940805] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_misc}
time:   00:43:09
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940805/

[6940806] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/damage}
time:   00:22:37
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940806/

[6940807] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_o_trunc}
time:   00:22:53
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940807/

[6940808] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:47:20
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940808/

[6940809] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/mon workloads/kclient_workunit_suites_ffsb}
time:   00:36:52
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940809/

[6940810] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/data-scan}
time:   00:27:48
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940810/

[6940811] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_snaps}
time:   00:47:50
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940811/

[6940812] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/failover}
time:   00:31:45
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940812/

[6940814] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub}
time:   00:16:35
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940814/

[6940815] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:28:00
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940815/

[6940816] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/default workloads/kclient_workunit_suites_iozone}
time:   00:27:25
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940816/

[6940817] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_ffsb}
time:   00:38:51
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940817/

[6940820] kcephfs/recovery/{begin clusters/1-mds-4-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush}
time:   00:17:32
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940820/

[6940823] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/no kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsx}
time:   00:22:46
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940823/

[6940825] kcephfs/mixed-clients/{begin clusters/1-mds-2-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:48:35
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940825/

[6940826] kcephfs/thrash/{begin clusters/1-mds-1-client conf/{client mds mon osd} kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{frag_enable log-config osd-asserts thrash-health-whitelist whitelist_health whitelist_wrongly_marked_down} thrashers/mds workloads/kclient_workunit_suites_ffsb}
time:   00:28:49
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940826/

[6940828] kcephfs/cephfs/{begin clusters/1-mds-1-client conf/{client mds mon osd} inline/yes kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{frag_enable log-config osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kclient_workunit_suites_fsync}
time:   00:13:02
info:   http://pulpito.ceph.com/yuriw-2022-07-20_14:56:20-kcephfs-octopus-release-testing-default-smithi/6940828/