2023-11-28T01:54:08.288 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5
2023-11-28T01:54:08.288 INFO:teuthology.results:subset: '111/12000'
2023-11-28T01:54:08.288 INFO:teuthology.results:seed: '3153'
2023-11-28T04:43:35.220 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5
2023-11-28T04:43:35.249 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-11-28T15:19:30.906 INFO:teuthology.results:Tests finished! gathering results...
2023-11-28T15:19:48.673 INFO:scrape:Found 173 jobs
2023-11-28T15:19:55.138 INFO:scrape:Found 7 distinct failure reasons
2023-11-28T15:19:55.139 INFO:scrape:Assertion: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1ad) [0x555ec4458a10]
2023-11-28T15:19:55.139 INFO:scrape:ceph version 16.2.14-270-g5474366c (5474366c84b0c469f7b5285fcfee86232293d8b4) pacific (stable)
 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f9189201420]
 2: gsignal()
 3: abort()
 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1ad) [0x555ec4458a10]
 5: ceph-osd(+0xad3bbd) [0x555ec4458bbd]
 6: (OSDService::~OSDService()+0x9a5) [0x555ec45d9465]
 7: (OSD::~OSD()+0x243) [0x555ec4577113]
 8: (OSD::~OSD()+0xd) [0x555ec4577a5d]
 9: main()
 10: __libc_start_main()
 11: _start()
2023-11-28T15:19:55.139 INFO:scrape:1 jobs: ['7468713']
2023-11-28T15:19:55.139 INFO:scrape:suites: ['clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'fs/verify/{begin', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{ignorelist_health', 'overrides}}', 'ranks/1', 'session_timeout}', 'tasks/dbench', 'validater/valgrind}']
2023-11-28T15:19:55.140 INFO:scrape:
2023-11-28T15:19:55.140 INFO:scrape:Failure: Command failed on smithi052 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5ea45e3a-8daa-11ee-95a2-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true'
2023-11-28T15:19:55.140 INFO:scrape:1 jobs: ['7468877']
2023-11-28T15:19:55.140 INFO:scrape:suites: ['1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '3-inline/no', '3-upgrade-with-workload', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn}', 'roles', 'tasks/{0-from/v16.2.4']
2023-11-28T15:19:55.140 INFO:scrape:
2023-11-28T15:19:55.140 INFO:scrape:Failure: Command failed on smithi016 with status 8: 'TESTDIR=/home/ubuntu/cephtest bash -s'
2023-11-28T15:19:55.140 INFO:scrape:2 jobs: ['7468757', '7468633']
2023-11-28T15:19:55.140 INFO:scrape:suites intersection: ['conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{ignorelist_health', 'tasks/ior-shared-file}']
2023-11-28T15:19:55.141 INFO:scrape:suites union: ['clusters/1-mds-2-client', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/ior-shared-file}']
2023-11-28T15:19:55.141 INFO:scrape:
2023-11-28T15:19:55.141 INFO:scrape:Failure: "2023-11-28T03:01:25.036550+0000 mon.a (mon.0) 334 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
2023-11-28T15:19:55.141 INFO:scrape:4 jobs: ['7468711', '7468750', '7468703', '7468847']
2023-11-28T15:19:55.141 INFO:scrape:suites intersection: ['conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}']
2023-11-28T15:19:55.141 INFO:scrape:suites union: ['clusters/1a11s-mds-1c-client-3node', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'fs/volumes/{begin', 'fs/workload/{begin', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'ms_mode/{legacy}', 'no_client_pidfile}', 'objectstore-ec/bluestore-comp', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'overrides/{ignorelist_health', 'ranks/3', 'rhel_8}', 'scrub/no', 'session_timeout}', 'standby-replay', 'subvolume/{no-subvolume}', 'tasks/metrics}', 'tasks/snap-schedule}', 'tasks/volumes/{overrides', 'tasks/{0-check-counter', 'test/basic}}', 'workunit/fs/test_o_trunc}', 'wsync/{yes}}']
2023-11-28T15:19:55.141 INFO:scrape:
2023-11-28T15:19:55.142 INFO:scrape:Failure: Command failed (workunit test fs/full/subvolume_clone.sh) on smithi089 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5474366c84b0c469f7b5285fcfee86232293d8b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/full/subvolume_clone.sh'
2023-11-28T15:19:55.142 INFO:scrape:1 jobs: ['7468720']
2023-11-28T15:19:55.142 INFO:scrape:suites: ['clusters/1-node-1-mds-1-osd', 'conf/{client', 'distro/{rhel_8}', 'fs/full/{begin', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides', 'overrides/{ignorelist_health', 'tasks/mgr-osd-full}']
2023-11-28T15:19:55.142 INFO:scrape:
2023-11-28T15:19:55.142 INFO:scrape:Failure: Test failure: test_acls (tasks.cephfs.test_acls.TestACLs)
2023-11-28T15:19:55.142 INFO:scrape:1 jobs: ['7468742']
2023-11-28T15:19:55.142 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/acls}']
2023-11-28T15:19:55.142 INFO:scrape:
2023-11-28T15:19:55.142 INFO:scrape:Dead: b'2023-11-28T15:11:36.644 DEBUG:teuthology.exit:Finished running handlers'
2023-11-28T15:19:55.143 INFO:scrape:1 jobs: ['7468756']
2023-11-28T15:19:55.143 INFO:scrape:suites: ['clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin', 'ignorelist_wrongly_marked_down', 'k-testing}', 'kclient-overrides/{distro/testing/{flavor/centos_latest', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-ec-root', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'tasks/kernel_cfuse_workunits_dbench_iozone}']
2023-11-28T15:19:55.143 INFO:scrape:
2023-11-28T15:19:55.151 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi
=================================================================
info:    https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/
failed:  10
dead:    1
running: 0
waiting: 0
queued:  0
passed:  162


Fail
=================================================================
[7468633]  fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file}
-----------------------------------------------------------------
time:   00:12:09
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468633/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468633/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cd0e6c43fa1849bba2c4f3f1dad56417

    Command failed on smithi122 with status 8: 'TESTDIR=/home/ubuntu/cephtest
    bash -s'

[7468703]  fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   01:45:54
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703/

    "2023-11-28T03:06:19.269245+0000 mon.a (mon.0) 594 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in
    cluster log

[7468711]  fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay subvolume/{no-subvolume} tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}}
-----------------------------------------------------------------
time:   00:24:31
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468711/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468711/

    "2023-11-28T03:01:25.036550+0000 mon.a (mon.0) 334 : cluster [WRN] Health
    check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in
    cluster log

[7468713]  fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind}
-----------------------------------------------------------------
time:   00:59:31
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468713/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468713/

    Found coredumps on ubuntu@smithi115.front.sepia.ceph.com

[7468720]  fs/full/{begin clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mgr-osd-full}
-----------------------------------------------------------------
time:   00:21:06
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468720/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468720/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1127982e2a39411db523ad59ee29492d

    Command failed (workunit test fs/full/subvolume_clone.sh) on smithi089 with
    status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=5474366c84b0c469f7b5285fcfee86232293d8b4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/fs/full/subvolume_clone.sh'

[7468742]  fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/acls}
-----------------------------------------------------------------
time:   00:16:11
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468742/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468742/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0b8ad701ad8e46509320c1853ec869ec

    Test failure: test_acls (tasks.cephfs.test_acls.TestACLs)

[7468750]  fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}
-----------------------------------------------------------------
time:   01:27:57
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468750/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468750/

    "2023-11-28T03:45:11.537290+0000 mon.a (mon.0) 4379 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in
    cluster log

[7468757]  fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file}
-----------------------------------------------------------------
time:   00:15:00
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468757/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468757/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e6d7a6d3d1fa402e96e4794db3b62a84

    Command failed on smithi016 with status 8: 'TESTDIR=/home/ubuntu/cephtest
    bash -s'

[7468847]  fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics}
-----------------------------------------------------------------
time:   00:31:59
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468847/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468847/

    "2023-11-28T04:47:14.102928+0000 mon.a (mon.0) 1601 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in
    cluster log

[7468877]  fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
-----------------------------------------------------------------
time:   00:13:53
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468877/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468877/
sentry: https://sentry.ceph.com/organizations/ceph/?query=66db5283d90d4d1ba4e64de552cf8d06

    Command failed on smithi052 with status 1: 'sudo
    /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c
    /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid
    5ea45e3a-8daa-11ee-95a2-87774f69a715 -- ceph config set mgr
    mgr/cephadm/allow_ptrace true'



Dead
=================================================================
[7468756]  fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468756/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468756/

    hit max job timeout



Pass
=================================================================
[7468631] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:33:44
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468631/

[7468635] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics}
time:   00:31:02
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468635/

[7468637] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}}
time:   00:14:22
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468637/

[7468639] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/fs/norstats} wsync/{no}}
time:   00:16:37
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468639/

[7468641] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc}
time:   00:23:54
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468641/

[7468643] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:07
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468643/

[7468646] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:24:15
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468646/

[7468648] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable}
time:   00:17:01
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468648/

[7468650] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}}
time:   00:24:31
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468650/

[7468652] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:15:18
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468652/

[7468654] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs_python}
time:   00:14:25
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468654/

[7468656] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:15:50
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468656/

[7468658] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}
time:   00:20:26
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468658/

[7468660] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}}
time:   00:18:15
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468660/

[7468662] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}
time:   00:15:09
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468662/

[7468663] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/lockdep}
time:   00:20:46
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468663/

[7468665] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:23:47
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468665/

[7468668] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:18:11
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468668/

[7468670] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}
time:   00:14:28
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468670/

[7468672] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/recovery-fs}
time:   00:17:57
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468672/

[7468674] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}}
time:   00:18:36
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468674/

[7468676] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:31:20
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468676/

[7468678] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:51:03
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468678/

[7468680] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc}
time:   00:17:32
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468680/

[7468682] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}}
time:   00:23:04
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468682/

[7468684] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:38
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468684/

[7468686] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub}
time:   00:30:36
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468686/

[7468688] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   01:00:58
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468688/

[7468690] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest}
time:   00:14:47
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468690/

[7468692] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:19:22
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468692/

[7468694] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap}
time:   00:16:09
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468694/

[7468696] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}}
time:   01:51:11
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468696/

[7468698] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:16:47
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468698/

[7468700] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:15:04
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468700/

[7468702] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}
time:   00:36:45
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468702/

[7468704] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}}
time:   00:24:02
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468704/

[7468705] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay subvolume/{with-no-extra-options} tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}}
time:   00:18:05
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468705/

[7468706] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}}
time:   00:27:40
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468706/

[7468707] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir}
time:   00:16:53
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468707/

[7468708] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots}
time:   00:29:01
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468708/

[7468709] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}}
time:   00:22:00
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468709/

[7468710] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:29:50
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468710/

[7468712] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
time:   00:28:25
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468712/

[7468714] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}}
time:   00:17:34
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468714/

[7468715] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration}
time:   00:19:11
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468715/

[7468716] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}}
time:   00:27:24
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468716/

[7468717] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:13
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468717/

[7468718] fs/bugs/client_trim_caps/{begin centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/trim-i24137}
time:   00:14:44
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468718/

[7468719] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/centos_latest}
time:   00:46:13
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468719/

[7468721] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client}
time:   00:13:41
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468721/

[7468722] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{centos_8} tasks/mirror}
time:   01:58:38
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468722/

[7468723] fs/mirror-ha/{begin cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{ubuntu_latest} workloads/cephfs-mirror-ha-workunit}
time:   02:37:44
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468723/

[7468724] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:29:46
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468724/

[7468725] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests}
time:   00:49:08
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468725/

[7468726] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:29:17
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468726/

[7468727] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:17:34
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468727/

[7468728] fs/shell/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell}
time:   00:37:41
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468728/

[7468729] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:56:40
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468729/

[7468730] fs/top/{begin cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/ignorelist_health supported-random-distros$/{centos_8} tasks/fstop}
time:   00:14:33
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468730/

[7468731] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc}
time:   00:22:47
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468731/

[7468732] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}}
time:   00:27:48
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468732/

[7468733] fs/valgrind/{begin centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}}
time:   02:10:32
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468733/

[7468734] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volume-client}
time:   00:29:26
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468734/

[7468735] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay}
time:   00:18:21
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468735/

[7468736] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/iogen} wsync/{no}}
time:   00:43:54
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468736/

[7468737] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}}
time:   01:44:27
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468737/

[7468738] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/dir-max-entries}
time:   00:17:29
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468738/

[7468739] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota}
time:   00:14:55
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468739/

[7468740] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}}
time:   00:16:34
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468740/

[7468741] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:59
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468741/

[7468743] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay subvolume/{with-no-extra-options} tasks/{0-check-counter workunit/suites/iozone} wsync/{yes}}
time:   00:19:35
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468743/

[7468744] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:18:41
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468744/

[7468745] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/admin}
time:   00:56:28
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468745/

[7468746] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-octopus 1-upgrade}}
time:   00:13:09
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468746/

[7468747] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}}
time:   00:19:07
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468747/

[7468748] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:14:27
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468748/

[7468749] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:14:10
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468749/

[7468751] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}
time:   00:15:23
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468751/

[7468752] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay subvolume/{no-subvolume} tasks/{0-check-counter workunit/suites/pjd} wsync/{no}}
time:   00:19:44
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468752/

[7468753] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:22:13
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468753/

[7468754] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/asok_dump_tree}
time:   00:25:11
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468754/

[7468755] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/fsstress validater/lockdep}
time:   00:20:16
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468755/

[7468758] fs/upgrade/upgraded_client/from_nautilus/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client-upgrade 2-client-sanity}}
time:   00:14:25
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468758/

[7468759] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/auto-repair}
time:   00:14:17
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468759/

[7468760] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}}
time:   00:16:53
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468760/

[7468761] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
time:   00:56:31
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468761/

[7468762] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:39:36
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468762/

[7468763] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc}
time:   00:47:53
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468763/

[7468764] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/backtrace}
time:   00:16:52
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468764/

[7468765] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:18:34
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468765/

[7468766] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay subvolume/{with-no-extra-options} tasks/{0-check-counter workunit/direct_io} wsync/{no}}
time:   00:17:50
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468766/

[7468767] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cap-flush}
time:   00:14:54
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468767/

[7468768] fs/upgrade/volumes/import-legacy/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04}
time:   01:13:48
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468768/

[7468769] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}}
time:   01:20:13
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468769/

[7468770] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:36
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468770/

[7468771] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/ino_release_cb}
time:   00:14:08
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468771/

[7468772] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:16:42
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468772/

[7468773] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}}
time:   01:00:31
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468773/

[7468774] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits}
time:   00:28:36
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468774/

[7468775] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}}
time:   00:37:12
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468775/

[7468776] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-readahead}
time:   00:13:07
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468776/

[7468777] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay subvolume/{with-no-extra-options} tasks/{0-check-counter workunit/fs/misc} wsync/{yes}}
time:   00:35:44
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468777/

[7468778] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:28:09
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468778/

[7468779] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery}
time:   00:52:32
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468779/

[7468780] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}}
time:   00:38:58
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468780/

[7468781] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage}
time:   00:29:19
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468781/

[7468782] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:33:32
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468782/

[7468783] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest}
time:   00:22:11
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468783/

[7468784] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}}
time:   00:18:13
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468784/

[7468785] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay subvolume/{with-quota} tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}}
time:   00:33:45
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468785/

[7468786] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind}
time:   00:55:04
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468786/

[7468787] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:24:35
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468787/

[7468788] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan}
time:   00:48:35
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468788/

[7468789] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}}
time:   00:22:44
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468789/

[7468790] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:15:06
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468790/

[7468791] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:28:06
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468791/

[7468792] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:16:10
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468792/

[7468793] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:39:22
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468793/

[7468794] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc}
time:   00:36:12
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468794/

[7468795] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}}
time:   00:14:57
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468795/

[7468796] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports}
time:   00:46:57
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468796/

[7468798] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}}
time:   01:19:17
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468798/

[7468800] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}
time:   00:21:18
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468800/

[7468802] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay subvolume/{with-namespace-isolated-and-quota} tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}}
time:   00:28:00
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468802/

[7468804] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:48
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468804/

[7468806] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/fragment}
time:   00:22:55
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468806/

[7468808] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}}
time:   00:15:09
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468808/

[7468810] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/journal-repair}
time:   00:20:36
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468810/

[7468812] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:21:21
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468812/

[7468814] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}}
time:   00:25:46
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468814/

[7468817] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay subvolume/{with-namespace-isolated-and-quota} tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}}
time:   00:46:19
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468817/

[7468819] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush}
time:   00:13:28
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468819/

[7468821] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}}
time:   00:25:13
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468821/

[7468823] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:16:11
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468823/

[7468825] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs/{frag test}}
time:   00:18:52
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468825/

[7468827] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:49:02
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468827/

[7468829] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests}
time:   00:48:58
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468829/

[7468831] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:14:27
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468831/

[7468833] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}
time:   00:20:07
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468833/

[7468835] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}
time:   00:28:16
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468835/

[7468837] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:19:13
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468837/

[7468838] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds_creation_retry}
time:   00:17:11
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468838/

[7468841] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:49
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468841/

[7468842] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay subvolume/{with-namespace-isolated-and-quota} tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}}
time:   00:34:35
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468842/

[7468845] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/fsstress validater/valgrind}
time:   01:46:49
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468845/

[7468849] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}}
time:   00:19:22
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468849/

[7468851] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
time:   00:54:06
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468851/

[7468852] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:50:30
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468852/

[7468855] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc}
time:   00:18:39
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468855/

[7468857] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc}
time:   00:19:43
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468857/

[7468859] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:16:31
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468859/

[7468861] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}}
time:   00:26:41
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468861/

[7468863] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable}
time:   00:18:24
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468863/

[7468865] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}}
time:   00:19:19
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468865/

[7468867] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}}
time:   01:47:46
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468867/

[7468869] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:46
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468869/

[7468871] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:18:44
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468871/

[7468873] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}
time:   00:31:23
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468873/

[7468875] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}
time:   00:15:24
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468875/

[7468879] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}}
time:   00:30:19
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468879/

[7468881] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}
time:   00:13:55
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468881/

[7468883] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:35:01
info:   https://pulpito.ceph.com/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468883/