2022-03-28T02:48:36.043 INFO:root:teuthology version: 1.1.0-70390753
2022-03-28T02:48:36.045 INFO:teuthology.results:subset: '111/120000'
2022-03-28T02:48:36.045 INFO:teuthology.results:seed: '2912'
2022-03-28T04:51:53.751 INFO:root:teuthology version: 1.1.0-70390753
2022-03-28T04:51:53.775 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-03-28T11:10:24.675 INFO:teuthology.results:Tests finished! gathering results...
2022-03-28T11:10:40.539 INFO:scrape:Found 180 jobs
2022-03-28T11:10:45.747 INFO:scrape:Found 6 distinct failure reasons
2022-03-28T11:10:45.748 INFO:scrape:Failure: "2022-03-28T03:28:56.232449+0000 mon.a (mon.0) 1759 : cluster [WRN] Health check failed: Degraded data redundancy: 3/184 objects degraded (1.630%), 2 pgs degraded (PG_DEGRADED)" in cluster log
2022-03-28T11:10:45.749 INFO:scrape:2 jobs: ['6764215', '6764344']
2022-03-28T11:10:45.749 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'mds', 'mon', 'osd}', 'overrides/{no_client_pidfile', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-03-28T11:10:45.750 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'fs/volumes/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{no_client_pidfile', 'tasks/strays}', 'tasks/volumes/{overrides', 'test/clone}}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-03-28T11:10:45.750 INFO:scrape:
2022-03-28T11:10:45.751 INFO:scrape:Dead: b'2022-03-28T11:07:41.045 DEBUG:teuthology.exit:Finished running handlers'
2022-03-28T11:10:45.751 INFO:scrape:2 jobs: ['6764356', '6764222']
2022-03-28T11:10:45.752 INFO:scrape:suites intersection: ['2-logrotate}', 'conf/{client', 'mds', 'mon', 'mount', 'objectstore-ec/bluestore-bitmap', 'osd}', 'session_timeout', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-03-28T11:10:45.752 INFO:scrape:suites union: ['1-ceph', '1-cephadm', '2-logrotate}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/verify/{begin/{0-install', 'fs/workload/{0-rhel_8', 'mds', 'mon', 'mount', 'mount/kclient/{k-testing', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'ms_mode/{crc}', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'overrides/{mon-debug', 'ranks/1', 'ranks/3', 'rhel_8}', 'scrub/no', 'session_timeout', 'standby-replay', 'tasks/fsstress', 'tasks/{0-check-counter', 'validater/valgrind}', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/suites/blogbench}', 'wsync/{no}}']
2022-03-28T11:10:45.752 INFO:scrape:
2022-03-28T11:10:45.753 INFO:scrape:Failure: Command failed on smithi083 with status 1: "(cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs)"
2022-03-28T11:10:45.753 INFO:scrape:1 jobs: ['6764371']
2022-03-28T11:10:45.754 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/suites/fsstress}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/thrash/workloads/{begin/{0-install', 'mds', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{frag', 'prefetch_dirfrags/no', 'races', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mon', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-03-28T11:10:45.754 INFO:scrape:
2022-03-28T11:10:45.755 INFO:scrape:Failure: Command failed on smithi033 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:14f44febaa74e8f8931e156f1a921292708ad47a -v bootstrap --fsid 17f27e68-ae4a-11ec-8c35-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.33 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring'
2022-03-28T11:10:45.755 INFO:scrape:5 jobs: ['6764272', '6764352', '6764309', '6764343', '6764230']
2022-03-28T11:10:45.756 INFO:scrape:suites intersection: ['1-cephadm', '2-logrotate}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{0-rhel_8', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'osd-asserts', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-03-28T11:10:45.756 INFO:scrape:suites union: ['1-cephadm', '2-logrotate}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'fs/workload/{0-rhel_8', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/{crc}', 'ms_mode/{legacy}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'ranks/1', 'ranks/5', 'scrub/no', 'scrub/yes', 'session_timeout', 'standby-replay', 'tasks/{0-check-counter', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'workunit/direct_io}', 'workunit/fs/test_o_trunc}', 'workunit/suites/dbench}', 'workunit/suites/fsstress}', 'workunit/suites/pjd}', 'wsync/{no}}', 'wsync/{yes}}']
2022-03-28T11:10:45.757 INFO:scrape:
2022-03-28T11:10:45.757 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds
2022-03-28T11:10:45.757 INFO:scrape:1 jobs: ['6764350']
2022-03-28T11:10:45.758 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/workloads/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'prefetch_dirfrags/no', 'races', 'ranks/1', 'rhel_8}', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}']
2022-03-28T11:10:45.758 INFO:scrape:
2022-03-28T11:10:45.759 INFO:scrape:Failure: Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.test_mirroring.TestMirroring)
2022-03-28T11:10:45.759 INFO:scrape:1 jobs: ['6764252']
2022-03-28T11:10:45.760 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'centos_latest', 'clients/mirror', 'cluster/1-node', 'debug', 'fs/valgrind/{begin/{0-install', 'mirror/{cephfs-mirror/one-per-cluster', 'mount/fuse', 'overrides/whitelist_health', 'tasks/mirror}}']
2022-03-28T11:10:45.760 INFO:scrape:
2022-03-28T11:10:45.768 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/
failed:  10
dead:    2
running: 0
waiting: 0
queued:  0
passed:  168


Fail
=================================================================
[6764215]  fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}}
-----------------------------------------------------------------
time:   00:40:05
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764215/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764215/

    "2022-03-28T03:28:56.232449+0000 mon.a (mon.0) 1759 : cluster [WRN] Health
    check failed: Degraded data redundancy: 3/184 objects degraded (1.630%), 2
    pgs degraded (PG_DEGRADED)" in cluster log

[6764230]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{no}}
-----------------------------------------------------------------
time:   00:10:44
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764230/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764230/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cb740b1edc6e46a9b0881d662e5633df

    Command failed on smithi017 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:14f44febaa74e8f8931e156f1a921292708ad47a -v bootstrap --fsid
    8b3f86ee-ae46-11ec-8c35-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-
    daemons --skip-monitoring-stack --mon-ip 172.21.15.17 --skip-admin-label &&
    sudo chmod +r /etc/ceph/ceph.client.admin.keyring'

[6764252]  fs/valgrind/{begin/{0-install 1-ceph 2-logrotate} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}}
-----------------------------------------------------------------
time:   01:19:33
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764252/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764252/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c177c345ed1f424c8ff15d90b8b9964c

    Test failure: test_cephfs_mirror_restart_sync_on_blocklist
    (tasks.cephfs.test_mirroring.TestMirroring)

[6764272]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}}
-----------------------------------------------------------------
time:   00:10:45
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764272/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764272/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0888c341a425480996b92d24bb2f75f4

    Command failed on smithi033 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:14f44febaa74e8f8931e156f1a921292708ad47a -v bootstrap --fsid
    17f27e68-ae4a-11ec-8c35-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-
    daemons --skip-monitoring-stack --mon-ip 172.21.15.33 --skip-admin-label &&
    sudo chmod +r /etc/ceph/ceph.client.admin.keyring'

[6764309]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}}
-----------------------------------------------------------------
time:   00:10:46
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764309/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764309/
sentry: https://sentry.ceph.com/organizations/ceph/?query=71e18974afae46fc945a1783dbabbe68

    Command failed on smithi008 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:14f44febaa74e8f8931e156f1a921292708ad47a -v bootstrap --fsid
    0fc4b4aa-ae4e-11ec-8c35-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-
    daemons --skip-monitoring-stack --mon-ip 172.21.15.8 --skip-admin-label &&
    sudo chmod +r /etc/ceph/ceph.client.admin.keyring'

[6764343]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}}
-----------------------------------------------------------------
time:   00:10:05
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764343/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764343/
sentry: https://sentry.ceph.com/organizations/ceph/?query=de1715b5748f4ce5951dea1c6bc262ad

    Command failed on smithi107 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:14f44febaa74e8f8931e156f1a921292708ad47a -v bootstrap --fsid
    711191da-ae51-11ec-8c35-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-
    daemons --skip-monitoring-stack --mon-ip 172.21.15.107 --skip-admin-label
    && sudo chmod +r /etc/ceph/ceph.client.admin.keyring'

[6764344]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/strays}
-----------------------------------------------------------------
time:   00:25:32
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764344/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764344/

    "2022-03-28T04:42:40.775198+0000 mon.a (mon.0) 994 : cluster [WRN] Health
    check failed: Degraded data redundancy: 4/50 objects degraded (8.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6764350]  fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}}
-----------------------------------------------------------------
time:   00:18:33
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764350/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764350/
sentry: https://sentry.ceph.com/organizations/ceph/?query=836021bdf623435db8a1b02eebb42deb

    reached maximum tries (90) after waiting for 540 seconds

[6764352]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}}
-----------------------------------------------------------------
time:   00:10:27
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764352/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764352/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bf3df7ad5b944f14b447792a0bf00e4f

    Command failed on smithi077 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-
    ci/ceph:14f44febaa74e8f8931e156f1a921292708ad47a -v bootstrap --fsid
    40392d4c-ae52-11ec-8c35-001a4aab830c --config
    /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf
    --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key
    /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-
    daemons --skip-monitoring-stack --mon-ip 172.21.15.77 --skip-admin-label &&
    sudo chmod +r /etc/ceph/ceph.client.admin.keyring'

[6764371]  fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}}
-----------------------------------------------------------------
time:   00:07:08
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764371/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764371/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7c647fad9978413e943a7b5d12db7bb1

    Command failed on smithi083 with status 1: "(cd /home/ubuntu/cephtest &&
    exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-
    mnt.admin sudo adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f
    --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok'
    /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs)"



Dead
=================================================================
[6764222]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764222/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764222/

    hit max job timeout

[6764356]  fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/fsstress validater/valgrind}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764356/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764356/

    hit max job timeout



Pass
=================================================================
[6764205] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota}
time:   00:11:51
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764205/

[6764206] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:12:14
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764206/

[6764207] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}}
time:   01:57:59
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764207/

[6764208] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   01:01:02
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764208/

[6764209] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc}
time:   00:14:09
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764209/

[6764210] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/fsstress validater/lockdep}
time:   00:15:54
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764210/

[6764211] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls}
time:   00:11:33
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764211/

[6764212] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/mdtest}
time:   00:12:24
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764212/

[6764213] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover}
time:   00:50:50
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764213/

[6764214] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:59:36
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764214/

[6764216] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:29:40
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764216/

[6764217] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}}
time:   01:08:28
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764217/

[6764218] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin}
time:   00:55:06
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764218/

[6764219] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}}
time:   02:57:24
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764219/

[6764220] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:15:19
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764220/

[6764221] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:10:56
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764221/

[6764223] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:19:22
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764223/

[6764224] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool}
time:   00:10:39
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764224/

[6764225] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}}
time:   01:49:27
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764225/

[6764226] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree}
time:   00:19:38
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764226/

[6764227] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:18
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764227/

[6764228] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair}
time:   00:14:10
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764228/

[6764229] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}}
time:   00:17:57
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764229/

[6764231] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/backtrace}
time:   00:10:24
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764231/

[6764232] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}}
time:   00:23:56
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764232/

[6764233] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}}
time:   00:22:00
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764233/

[6764234] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cap-flush}
time:   00:10:00
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764234/

[6764235] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}}
time:   01:26:11
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764235/

[6764236] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:13:55
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764236/

[6764237] fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate} centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/trim-i24137}
time:   00:11:13
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764237/

[6764238] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/centos_latest}
time:   00:43:49
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764238/

[6764239] fs/full/{begin/{0-install 1-ceph 2-logrotate} clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mgr-osd-full}
time:   00:14:23
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764239/

[6764240] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client}
time:   00:09:51
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764240/

[6764241] fs/mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{ubuntu_latest} tasks/mirror}
time:   01:54:26
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764241/

[6764242] fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/three-per-cluster clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{rhel_8} workloads/cephfs-mirror-ha-workunit}
time:   00:20:33
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764242/

[6764243] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:39:06
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764243/

[6764244] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests}
time:   00:19:36
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764244/

[6764245] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth}
time:   00:26:56
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764245/

[6764246] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:12:47
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764246/

[6764247] fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/cephfs-shell}
time:   00:33:13
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764247/

[6764248] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:46:01
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764248/

[6764249] fs/top/{begin/{0-install 1-ceph 2-logrotate} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/whitelist_health supported-random-distros$/{ubuntu_latest} tasks/fstop}
time:   00:08:19
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764249/

[6764250] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc}
time:   00:20:00
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764250/

[6764251] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:27:47
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764251/

[6764253] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}}
time:   00:14:32
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764253/

[6764254] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-limits}
time:   00:19:23
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764254/

[6764255] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}}
time:   00:57:30
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764255/

[6764256] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/dbench validater/valgrind}
time:   00:49:19
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764256/

[6764257] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}}
time:   00:10:59
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764257/

[6764258] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-readahead}
time:   00:09:53
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764258/

[6764259] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:28:46
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764259/

[6764260] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/client-recovery}
time:   00:28:34
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764260/

[6764261] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:19:59
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764261/

[6764262] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}}
time:   00:23:46
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764262/

[6764263] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/damage}
time:   00:22:19
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764263/

[6764264] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-upgrade}}
time:   00:10:08
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764264/

[6764265] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}}
time:   00:21:58
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764265/

[6764266] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/data-scan}
time:   00:26:08
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764266/

[6764267] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:14:14
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764267/

[6764268] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:18:02
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764268/

[6764269] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:12:24
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764269/

[6764270] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:12:59
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764270/

[6764271] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/exports}
time:   00:41:50
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764271/

[6764273] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/forward-scrub}
time:   00:17:12
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764273/

[6764274] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}}
time:   00:13:49
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764274/

[6764275] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:21
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764275/

[6764276] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file}
time:   00:10:16
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764276/

[6764277] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover}
time:   00:49:39
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764277/

[6764278] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:49:11
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764278/

[6764279] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}}
time:   00:19:18
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764279/

[6764280] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/fragment}
time:   00:16:07
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764280/

[6764281] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:13:52
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764281/

[6764282] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:50:52
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764282/

[6764283] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc}
time:   00:48:01
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764283/

[6764284] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/journal-repair}
time:   00:16:32
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764284/

[6764285] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}}
time:   00:24:30
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764285/

[6764286] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}}
time:   00:34:57
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764286/

[6764287] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}}
time:   01:50:42
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764287/

[6764288] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/fsstress validater/lockdep}
time:   00:16:16
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764288/

[6764289] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-flush}
time:   00:12:00
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764289/

[6764290] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:28:49
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764290/

[6764291] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/ffsb}}
time:   00:55:49
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764291/

[6764292] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:14:18
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764292/

[6764293] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds-full}
time:   00:18:41
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764293/

[6764294] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/ino_release_cb}
time:   00:10:39
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764294/

[6764295] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:11:04
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764295/

[6764296] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/mds_creation_retry}
time:   00:12:17
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764296/

[6764297] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}}
time:   00:21:14
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764297/

[6764298] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}}
time:   02:31:16
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764298/

[6764299] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:27:25
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764299/

[6764300] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/metrics}
time:   00:23:58
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764300/

[6764301] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/iozone}}
time:   00:13:41
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764301/

[6764302] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/multimds_misc}
time:   00:16:46
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764302/

[6764303] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:50
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764303/

[6764304] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/mdtest}
time:   00:19:40
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764304/

[6764305] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth}
time:   00:22:51
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764305/

[6764306] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:37:42
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764306/

[6764307] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}}
time:   00:56:47
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764307/

[6764308] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}}
time:   00:28:08
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764308/

[6764310] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/openfiletable}
time:   00:11:33
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764310/

[6764311] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:18:36
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764311/

[6764312] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}}
time:   01:08:00
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764312/

[6764313] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/pool-perm}
time:   00:12:05
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764313/

[6764314] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:11:43
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764314/

[6764315] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:51:26
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764315/

[6764316] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:11:44
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764316/

[6764317] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc}
time:   00:56:41
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764317/

[6764318] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/quota}
time:   00:10:56
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764318/

[6764319] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}}
time:   00:10:28
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764319/

[6764320] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:27:45
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764320/

[6764321] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}}
time:   00:35:11
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764321/

[6764322] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/recovery-fs}
time:   00:09:33
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764322/

[6764323] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{mon-debug session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/dbench validater/valgrind}
time:   00:50:30
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764323/

[6764324] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:16:48
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764324/

[6764325] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}}
time:   00:25:42
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764325/

[6764326] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/scrub}
time:   00:21:22
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764326/

[6764327] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{frag prefetch_dirfrags/no races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}}
time:   00:16:54
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764327/

[6764328] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/sessionmap}
time:   00:14:44
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764328/

[6764329] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{no}}
time:   00:23:00
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764329/

[6764330] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:16
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764330/

[6764331] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule}
time:   00:29:14
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764331/

[6764332] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:13:45
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764332/

[6764333] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:11:10
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764333/

[6764334] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs/{frag test}}
time:   00:12:20
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764334/

[6764335] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests}
time:   00:19:35
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764335/

[6764336] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/failover}
time:   00:46:56
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764336/

[6764337] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:11:46
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764337/

[6764338] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps}
time:   01:21:15
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764338/

[6764339] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}}
time:   00:38:40
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764339/

[6764340] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snapshots}
time:   00:22:13
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764340/

[6764341] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:25:33
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764341/

[6764342] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}}
time:   00:12:08
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764342/

[6764345] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:14:15
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764345/

[6764346] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:52:33
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764346/

[6764347] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc}
time:   00:15:38
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764347/

[6764348] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-upgrade}}
time:   00:10:48
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764348/

[6764349] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/test_journal_migration}
time:   00:14:30
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764349/

[6764351] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/truncate_delay}
time:   00:13:36
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764351/

[6764353] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:28:00
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764353/

[6764354] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/dir-max-entries}
time:   00:11:56
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764354/

[6764355] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mon 2-workunit/ffsb}}
time:   00:57:28
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764355/

[6764357] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:15:38
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764357/

[6764358] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:12:37
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764358/

[6764359] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/workunit/quota}
time:   00:11:40
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764359/

[6764360] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:18:43
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764360/

[6764361] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}}
time:   02:01:18
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764361/

[6764362] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/acls}
time:   00:15:58
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764362/

[6764363] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/ior-shared-file}
time:   00:09:57
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764363/

[6764364] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{mon-debug whitelist_health whitelist_wrongly_marked_down} tasks/multifs-auth}
time:   00:20:21
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764364/

[6764365] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:32:57
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764365/

[6764366] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}}
time:   00:15:14
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764366/

[6764367] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}}
time:   01:06:22
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764367/

[6764368] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/iozone}}
time:   00:14:30
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764368/

[6764369] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:14
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764369/

[6764370] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/admin}
time:   00:50:35
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764370/

[6764372] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/alternate-pool}
time:   00:10:28
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764372/

[6764373] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}}
time:   00:24:29
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764373/

[6764374] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}}
time:   00:42:29
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764374/

[6764375] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/asok_dump_tree}
time:   00:22:39
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764375/

[6764376] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag multifs session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}}
time:   01:16:50
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764376/

[6764377] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:12:13
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764377/

[6764378] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python}
time:   00:09:24
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764378/

[6764379] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{osd-asserts whitelist_health whitelist_wrongly_marked_down} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:27:29
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764379/

[6764380] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:12:07
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764380/

[6764381] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc}
time:   00:20:52
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764381/

[6764382] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/auto-repair}
time:   00:11:10
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764382/

[6764383] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:26:00
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764383/

[6764384] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{frag prefetch_dirfrags/yes races session_timeout thrashosds-health whitelist_health whitelist_wrongly_marked_down} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}}
time:   00:11:15
info:   http://pulpito.ceph.com/yuriw-2022-03-27_21:53:11-fs-quincy-distro-default-smithi/6764384/