2023-07-23T23:56:10.851 INFO:root:teuthology version: 0.0.1.dev158+g1426592
2023-07-23T23:56:10.852 INFO:teuthology.results:subset: '111/12000'
2023-07-23T23:56:10.852 INFO:teuthology.results:seed: '5791'
2023-07-24T02:45:49.878 INFO:root:teuthology version: 0.0.1.dev158+g1426592
2023-07-24T02:45:49.912 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-07-24T13:35:42.579 INFO:teuthology.results:Tests finished! gathering results...
2023-07-24T13:35:59.902 INFO:scrape:Found 194 jobs
2023-07-24T13:36:09.783 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348935/teuthology.log
2023-07-24T13:36:10.222 INFO:scrape:Found 9 distinct failure reasons
2023-07-24T13:36:10.223 INFO:scrape:Failure: Command failed (workunit test libcephfs/test.sh) on smithi143 with status 139: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c2f10834dee12ea9c20f072e7ad4b3d7cb4d5f63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh'
2023-07-24T13:36:10.223 INFO:scrape:1 jobs: ['7348912']
2023-07-24T13:36:10.224 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/{ubuntu_20.04}', 'fs/libcephfs/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/libcephfs/{frag', 'test}}']
2023-07-24T13:36:10.224 INFO:scrape:
2023-07-24T13:36:10.224 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi049 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c2f10834dee12ea9c20f072e7ad4b3d7cb4d5f63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-multiple-capsnaps.sh'
2023-07-24T13:36:10.225 INFO:scrape:2 jobs: ['7348916', '7348932']
2023-07-24T13:36:10.225 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'conf/{client', 'mds', 'mon', 'mount/fuse', 'osd}']
2023-07-24T13:36:10.226 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', '2-workunit/cfuse_workunit_snaptests}}', 'clusters/1a3s-mds-1c-client', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/snaps/{begin/{0-install', 'fs/thrash/multifs/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag', 'overrides/{ignorelist_health', 'session_timeout', 'tasks/workunit/snaps}', 'tasks/{1-thrash/mon', 'thrashosds-health}']
2023-07-24T13:36:10.226 INFO:scrape:
2023-07-24T13:36:10.226 INFO:scrape:Failure: reached maximum tries (51) after waiting for 300 seconds
2023-07-24T13:36:10.227 INFO:scrape:1 jobs: ['7349010']
2023-07-24T13:36:10.227 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'cephfs-mirror/three-per-cluster', 'clients/{mirror}', 'cluster/{1-node}', 'fs/mirror-ha/{begin/{0-install', 'objectstore/bluestore-bitmap', 'overrides/{whitelist_health}', 'supported-random-distro$/{ubuntu_20.04}', 'workloads/cephfs-mirror-ha-workunit}']
2023-07-24T13:36:10.227 INFO:scrape:
2023-07-24T13:36:10.227 INFO:scrape:Dead: b'2023-07-24T13:33:01.736 DEBUG:teuthology.exit:Finished running handlers'
2023-07-24T13:36:10.228 INFO:scrape:1 jobs: ['7349005']
2023-07-24T13:36:10.228 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'bluestore-bitmap', 'clusters/1-mds-1-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/fscrypt/{begin/{0-install', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/v1', 'osd}', 'overrides/{distro/testing/k-testing}}', 'overrides/{ignorelist_health', 'pg-warn}', 'tasks/fscrypt-common}']
2023-07-24T13:36:10.228 INFO:scrape:
2023-07-24T13:36:10.229 INFO:scrape:Failure: "2023-07-24T00:49:23.387372+0000 mon.a (mon.0) 546 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 5 pgs peering (PG_AVAILABILITY)" in cluster log
2023-07-24T13:36:10.229 INFO:scrape:1 jobs: ['7348943']
2023-07-24T13:36:10.229 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/snap-schedule}']
2023-07-24T13:36:10.230 INFO:scrape:
2023-07-24T13:36:10.230 INFO:scrape:Failure: Command failed on smithi092 with status 2: 'TESTDIR=/home/ubuntu/cephtest bash -s'
2023-07-24T13:36:10.230 INFO:scrape:5 jobs: ['7349044', '7348940', '7348946', '7348977', '7349074']
2023-07-24T13:36:10.231 INFO:scrape:suites intersection: ['2-logrotate}', 'conf/{client', 'mds', 'mon', 'osd}']
2023-07-24T13:36:10.231 INFO:scrape:suites union: ['1-ceph', '1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', '5-workunit/suites/iogen}}', 'begin/{0-install', 'clusters/1-mds-2-client', 'clusters/1-mds-3-client', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distros/ubuntu_latest', 'frag', 'fs/multiclient/{begin/{0-install', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/secure', 'n/3', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'ranks/multi/{export-check', 'replication/always}', 'rhel_8}', 'session_timeout}', 'standby-replay', 'tasks/ior-shared-file}', 'tasks/mdtest}', 'tasks/{0-subvolume/{with-quota}', 'wsync/yes}']
2023-07-24T13:36:10.232 INFO:scrape:
2023-07-24T13:36:10.233 INFO:scrape:Failure: Command failed (workunit test kernel_untar_build.sh) on smithi145 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c2f10834dee12ea9c20f072e7ad4b3d7cb4d5f63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh'
2023-07-24T13:36:10.234 INFO:scrape:1 jobs: ['7348937']
2023-07-24T13:36:10.234 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'overrides}}', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}']
2023-07-24T13:36:10.235 INFO:scrape:
2023-07-24T13:36:10.236 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi082 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c2f10834dee12ea9c20f072e7ad4b3d7cb4d5f63 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-git-ceph.sh'
2023-07-24T13:36:10.236 INFO:scrape:1 jobs: ['7348999']
2023-07-24T13:36:10.237 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/cfuse_workunit_snaptests}}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{ubuntu_20.04}', 'fs/thrash/multifs/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}']
2023-07-24T13:36:10.237 INFO:scrape:
2023-07-24T13:36:10.237 INFO:scrape:Dead: None
2023-07-24T13:36:10.238 INFO:scrape:1 jobs: ['7348935']
2023-07-24T13:36:10.238 INFO:scrape:
2023-07-24T13:36:10.248 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/
failed:  12
dead:    2
running: 0
waiting: 0
queued:  0
passed:  180


Fail
=================================================================
[7348912]  fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_20.04} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs/{frag test}}
-----------------------------------------------------------------
time:   00:21:25
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348912/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348912/
sentry: https://sentry.ceph.com/organizations/ceph/?query=57ca024b1f2d4e2e96d02e508b2eda82

    Command failed (workunit test libcephfs/test.sh) on smithi143 with status
    139: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2f10834dee12ea9c20f072e7ad4b3d7cb4d5f63
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh'

[7348916]  fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
-----------------------------------------------------------------
time:   00:40:22
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348916/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348916/
sentry: https://sentry.ceph.com/organizations/ceph/?query=125a5cb519c7403d8c42d381554c1fc6

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on
    smithi049 with status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2f10834dee12ea9c20f072e7ad4b3d7cb4d5f63
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-
    multiple-capsnaps.sh'

[7348932]  fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}}
-----------------------------------------------------------------
time:   00:50:16
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348932/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348932/
sentry: https://sentry.ceph.com/organizations/ceph/?query=80ab1fa2e570485a8d5b03efb430aeaa

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on
    smithi087 with status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2f10834dee12ea9c20f072e7ad4b3d7cb4d5f63
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-
    multiple-capsnaps.sh'

[7348937]  fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
-----------------------------------------------------------------
time:   00:28:44
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348937/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348937/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ae89174f66ff4b8f909c12340f60decb

    Command failed (workunit test kernel_untar_build.sh) on smithi145 with
    status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd --
    /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2f10834dee12ea9c20f072e7ad4b3d7cb4d5f63
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1
    CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.1/qa/workunits/kernel_untar_build.sh'

[7348940]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/iogen}}
-----------------------------------------------------------------
time:   00:17:51
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348940/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348940/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b2ba3e3c31ea4667abff5f9ff9dc2e17

    Command failed on smithi033 with status 5: 'sudo systemctl stop
    ceph-28a3d96c-29bb-11ee-9b34-001a4aab830c@mon.b'

[7348943]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   00:52:54
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348943/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348943/

    "2023-07-24T00:49:23.387372+0000 mon.a (mon.0) 546 : cluster [WRN] Health
    check failed: Reduced data availability: 2 pgs inactive, 5 pgs peering
    (PG_AVAILABILITY)" in cluster log

[7348946]  fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file}
-----------------------------------------------------------------
time:   00:15:05
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348946/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348946/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cc136848c2db475dafa9db22b9c24ea0

    Command failed on smithi086 with status 2: 'TESTDIR=/home/ubuntu/cephtest
    bash -s'

[7348977]  fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest}
-----------------------------------------------------------------
time:   00:18:13
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348977/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348977/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d54454241bbf4ea0acef642a9cad6586

    Command failed on smithi066 with status 2: 'TESTDIR=/home/ubuntu/cephtest
    bash -s'

[7348999]  fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}}
-----------------------------------------------------------------
time:   00:50:26
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348999/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348999/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2f0cb5334a2543c8be45752bbfa6a2c3

    Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi082
    with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp &&
    cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2f10834dee12ea9c20f072e7ad4b3d7cb4d5f63
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1
    CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h
    /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-git-
    ceph.sh'

[7349010]  fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/three-per-cluster clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{ubuntu_20.04} workloads/cephfs-mirror-ha-workunit}
-----------------------------------------------------------------
time:   00:32:11
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349010/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349010/

    reached maximum tries (51) after waiting for 300 seconds

[7349044]  fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file}
-----------------------------------------------------------------
time:   00:15:42
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349044/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349044/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bdec470c5cf24a30964f930936510dc5

    Command failed on smithi092 with status 2: 'TESTDIR=/home/ubuntu/cephtest
    bash -s'

[7349074]  fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest}
-----------------------------------------------------------------
time:   00:15:00
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349074/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349074/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e02c86c830d64fe0a57f0726f328f5fc

    Command failed on smithi121 with status 2: 'TESTDIR=/home/ubuntu/cephtest
    bash -s'



Dead
=================================================================
[7348935]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348935/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348935/

    Error reimaging machines: Failed to power on smithi173

[7349005]  fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/v1 mount overrides/{distro/testing/k-testing}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/fscrypt-common}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349005/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349005/

    hit max job timeout



Pass
=================================================================
[7348908] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc}
time:   00:23:33
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348908/

[7348909] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:16:33
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348909/

[7348910] fs/cephadm/renamevolume/{0-start 1-rename distro/single-container-host overrides/ignorelist_health}
time:   00:44:39
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348910/

[7348911] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable}
time:   00:40:30
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348911/

[7348913] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests}
time:   01:01:49
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348913/

[7348914] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
time:   00:59:51
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348914/

[7348915] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:18:39
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348915/

[7348917] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}}
time:   00:23:51
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348917/

[7348918] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/fsync-tester}}
time:   00:40:29
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348918/

[7348919] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   01:02:03
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348919/

[7348920] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}
time:   00:20:24
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348920/

[7348921] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}
time:   00:40:48
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348921/

[7348922] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}}
time:   00:24:51
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348922/

[7348923] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-pacific 1-upgrade}}
time:   00:31:54
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348923/

[7348924] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}
time:   00:33:30
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348924/

[7348925] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/dbench validater/valgrind}
time:   00:57:51
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348925/

[7348926] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-workload/new_ops/{0-clients/kclient 1-client-sanity}}}
time:   00:26:10
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348926/

[7348927] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}}
time:   00:23:42
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348927/

[7348928] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/fs/test_o_trunc}}
time:   00:57:47
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348928/

[7348929] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/recovery-fs}
time:   00:13:09
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348929/

[7348930] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:47:28
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348930/

[7348931] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub}
time:   00:25:14
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348931/

[7348933] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/v1 mount overrides/{distro/testing/k-testing}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/fscrypt-iozone}
time:   02:38:17
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348933/

[7348934] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/iozone}}}
time:   00:29:31
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348934/

[7348936] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:58
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348936/

[7348938] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:18:54
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348938/

[7348939] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc}
time:   00:53:57
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348939/

[7348941] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:32:56
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348941/

[7348942] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}}
time:   00:26:13
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348942/

[7348944] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:21:51
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348944/

[7348945] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir}
time:   00:14:57
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348945/

[7348947] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:29:06
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348947/

[7348948] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:38:44
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348948/

[7348949] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}
time:   02:05:01
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348949/

[7348950] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iozone}}
time:   00:28:21
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348950/

[7348951] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}}
time:   00:17:03
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348951/

[7348952] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots}
time:   00:30:24
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348952/

[7348953] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}}
time:   00:37:59
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348953/

[7348954] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
time:   00:30:47
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348954/

[7348955] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:32:13
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348955/

[7348956] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/blogbench}}}
time:   00:38:15
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348956/

[7348957] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration}
time:   00:27:09
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348957/

[7348958] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/pjd}}
time:   00:37:21
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348958/

[7348959] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:25:05
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348959/

[7348960] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs_python}
time:   00:22:18
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348960/

[7348961] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:25:11
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348961/

[7348962] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}}
time:   01:16:59
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348962/

[7348963] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:43:56
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348963/

[7348964] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/lockdep}
time:   00:26:09
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348964/

[7348965] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay}
time:   00:28:21
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348965/

[7348966] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:41:29
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348966/

[7348967] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:25:32
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348967/

[7348968] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/dir-max-entries}
time:   00:26:13
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348968/

[7348969] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/direct_io}}
time:   00:49:08
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348969/

[7348970] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/v1 mount overrides/{distro/testing/k-testing}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/fscrypt-pjd}
time:   00:29:00
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348970/

[7348971] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}}
time:   00:58:33
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348971/

[7348972] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota}
time:   00:26:04
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348972/

[7348973] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:58:59
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348973/

[7348974] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc}
time:   00:26:09
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348974/

[7348975] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}}
time:   00:45:30
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348975/

[7348976] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/acls}
time:   00:20:43
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348976/

[7348978] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
time:   00:56:08
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348978/

[7348979] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   01:02:42
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348979/

[7348980] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}}
time:   00:30:12
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348980/

[7348981] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/misc}}
time:   00:57:20
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348981/

[7348982] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/admin}
time:   01:13:48
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348982/

[7348983] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:27:58
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348983/

[7348984] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}}
time:   00:27:34
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348984/

[7348985] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:24:42
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348985/

[7348986] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:22:23
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348986/

[7348987] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}
time:   00:14:33
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348987/

[7348988] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/asok_dump_tree}
time:   00:29:12
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348988/

[7348989] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}}
time:   00:21:06
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348989/

[7348990] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/kernel_untar_build}}
time:   02:24:37
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348990/

[7348991] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/auto-repair}
time:   00:17:27
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348991/

[7348992] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}}
time:   00:17:56
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348992/

[7348993] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:34:11
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348993/

[7348994] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}}
time:   00:21:32
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348994/

[7348995] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/backtrace}
time:   00:15:33
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348995/

[7348996] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}}
time:   00:26:44
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348996/

[7348997] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind}
time:   01:04:39
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348997/

[7348998] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}}
time:   00:58:20
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7348998/

[7349000] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cap-flush}
time:   00:17:27
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349000/

[7349001] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}}
time:   00:19:28
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349001/

[7349002] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:24:59
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349002/

[7349003] fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate} centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/trim-i24137}
time:   00:15:26
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349003/

[7349004] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/single-container-host}
time:   00:54:05
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349004/

[7349006] fs/full/{begin/{0-install 1-ceph 2-logrotate} clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mgr-osd-full}
time:   00:21:55
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349006/

[7349007] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits}
time:   00:25:41
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349007/

[7349008] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client}
time:   00:14:20
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349008/

[7349009] fs/mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{rhel_8} tasks/mirror}
time:   02:05:30
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349009/

[7349011] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:45:20
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349011/

[7349012] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests}
time:   01:00:17
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349012/

[7349013] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:36:34
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349013/

[7349014] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:16:17
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349014/

[7349015] fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell}
time:   00:43:54
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349015/

[7349016] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:59:22
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349016/

[7349017] fs/top/{begin/{0-install 1-ceph 2-logrotate} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/ignorelist_health supported-random-distros$/{ubuntu_20.04} tasks/fstop}
time:   00:14:59
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349017/

[7349018] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc}
time:   00:24:29
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349018/

[7349019] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:34:48
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349019/

[7349020] fs/valgrind/{begin/{0-install 1-ceph 2-logrotate} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}}
time:   02:12:01
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349020/

[7349021] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:35:14
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349021/

[7349022] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/blogbench}}
time:   00:52:04
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349022/

[7349023] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-readahead}
time:   00:18:41
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349023/

[7349024] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:21:02
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349024/

[7349025] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-pacific 1-upgrade}}
time:   00:13:27
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349025/

[7349026] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery}
time:   00:50:08
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349026/

[7349027] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}}
time:   00:16:46
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349027/

[7349028] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/new_ops/{0-clients/fuse-upgrade 1-client-sanity}}}
time:   00:14:07
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349028/

[7349029] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}
time:   00:24:41
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349029/

[7349030] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage}
time:   00:36:37
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349030/

[7349031] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}}
time:   01:20:33
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349031/

[7349032] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:20:10
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349032/

[7349033] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan}
time:   00:46:57
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349033/

[7349034] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:27:29
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349034/

[7349035] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports}
time:   00:57:34
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349035/

[7349036] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}}
time:   01:21:37
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349036/

[7349037] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:22:49
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349037/

[7349038] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:21:36
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349038/

[7349039] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}}
time:   00:58:36
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349039/

[7349040] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}
time:   00:18:49
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349040/

[7349041] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:18:43
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349041/

[7349042] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/fsstress validater/lockdep}
time:   00:21:17
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349042/

[7349043] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/fragment}
time:   00:26:41
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349043/

[7349045] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
time:   01:04:59
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349045/

[7349046] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:55:48
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349046/

[7349047] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/fsstress}}}
time:   00:20:58
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349047/

[7349048] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}}
time:   00:29:37
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349048/

[7349049] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:44:01
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349049/

[7349050] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc}
time:   00:47:41
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349050/

[7349051] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/journal-repair}
time:   00:22:46
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349051/

[7349052] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/norstats}}
time:   00:31:23
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349052/

[7349053] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}}
time:   00:37:22
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349053/

[7349054] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/v1 mount overrides/{distro/testing/k-testing}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/fscrypt-dbench}
time:   01:10:32
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349054/

[7349055] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}
time:   00:40:42
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349055/

[7349056] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}}
time:   00:50:18
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349056/

[7349057] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush}
time:   00:12:53
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349057/

[7349058] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:31:05
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349058/

[7349059] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}
time:   00:24:55
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349059/

[7349060] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}}
time:   00:26:06
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349060/

[7349061] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:20
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349061/

[7349062] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/ino_release_cb}
time:   00:14:03
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349062/

[7349063] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:17:57
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349063/

[7349064] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/fsstress}}
time:   00:42:55
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349064/

[7349065] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds_creation_retry}
time:   00:20:09
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349065/

[7349066] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}}
time:   00:26:25
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349066/

[7349067] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:35:42
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349067/

[7349068] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics}
time:   00:39:28
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349068/

[7349069] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}}
time:   00:17:52
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349069/

[7349070] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}}
time:   00:31:35
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349070/

[7349071] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc}
time:   00:21:04
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349071/

[7349072] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/kernel_untar_build}}}
time:   01:22:17
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349072/

[7349073] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable}
time:   00:17:27
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349073/

[7349075] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:35:40
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349075/

[7349076] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:51:49
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349076/

[7349077] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}
time:   01:48:00
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349077/

[7349078] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}}
time:   01:34:54
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349078/

[7349079] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:27:58
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349079/

[7349080] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind}
time:   00:56:37
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349080/

[7349081] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}
time:   00:16:01
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349081/

[7349082] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/fsync-tester}}
time:   00:41:40
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349082/

[7349083] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}}
time:   00:20:41
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349083/

[7349084] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:22:43
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349084/

[7349085] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:48:52
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349085/

[7349086] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:18:26
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349086/

[7349087] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc}
time:   00:38:36
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349087/

[7349088] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}
time:   00:15:28
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349088/

[7349089] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:33:34
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349089/

[7349090] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:22:06
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349090/

[7349091] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/v1 mount overrides/{distro/testing/k-testing}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/fscrypt-ffsb}
time:   02:09:32
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349091/

[7349092] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/recovery-fs}
time:   00:15:19
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349092/

[7349093] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}}
time:   00:35:34
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349093/

[7349094] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub}
time:   00:22:43
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349094/

[7349095] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}}
time:   00:19:04
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349095/

[7349096] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}}
time:   00:30:06
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349096/

[7349097] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap}
time:   00:17:33
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349097/

[7349098] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/dbench}}}
time:   01:00:44
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349098/

[7349099] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:19:47
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349099/

[7349100] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}}
time:   01:12:31
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349100/

[7349101] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
time:   00:53:24
info:   http://pulpito.ceph.com/yuriw-2023-07-23_14:06:30-fs-reef-distro-default-smithi/7349101/