2023-09-23T02:50:41.178 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-23T02:50:41.179 INFO:teuthology.results:subset: '57/1024'
2023-09-23T02:50:41.179 INFO:teuthology.results:seed: '8196'
2023-09-23T09:06:22.627 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-23T09:06:22.654 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-09-23T18:39:08.184 INFO:teuthology.results:Tests finished! gathering results...
2023-09-23T18:39:25.653 INFO:scrape:Found 201 jobs
2023-09-23T18:39:26.919 WARNING:scrape:Misunderstood line: 
2023-09-23T18:39:26.920 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401443/teuthology.log
2023-09-23T18:39:26.923 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401443/teuthology.log
2023-09-23T18:39:26.924 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401477/teuthology.log
2023-09-23T18:39:26.928 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401562/teuthology.log
2023-09-23T18:39:29.971 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401464/teuthology.log
2023-09-23T18:39:30.927 WARNING:scrape:Saw end of BT but not start
2023-09-23T18:39:30.927 WARNING:scrape:Saw end of BT but not start
2023-09-23T18:39:30.927 WARNING:scrape:Saw end of BT but not start
2023-09-23T18:39:30.927 WARNING:scrape:Saw end of BT but not start
2023-09-23T18:39:31.089 WARNING:scrape:Saw end of BT but not start
2023-09-23T18:39:31.090 WARNING:scrape:Saw end of BT but not start
2023-09-23T18:39:31.090 WARNING:scrape:Saw end of BT but not start
2023-09-23T18:39:31.090 WARNING:scrape:Saw end of BT but not start
2023-09-23T18:39:34.316 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401543/teuthology.log
2023-09-23T18:39:34.323 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401556/teuthology.log
2023-09-23T18:39:35.784 INFO:scrape:Found 11 distinct failure reasons
2023-09-23T18:39:35.784 INFO:scrape:Dead: b'2023-09-23T18:17:31.102 DEBUG:teuthology.exit:Finished running handlers'
2023-09-23T18:39:35.784 INFO:scrape:ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)
 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xd7) [0x7f0061eb9325]
 2: (MonitorDBStore::apply_transaction(std::shared_ptr<MonitorDBStore::Transaction>)+0x88f) [0x55cabb492e7f]
 3: (Paxos::begin(ceph::buffer::v15_2_0::list&)+0x413) [0x55cabb5a6193]
 4: (Paxos::propose_pending()+0x126) [0x55cabb5aad36]
 5: (Paxos::trigger_propose()+0x146) [0x55cabb5ab156]
 6: (PaxosService::propose_pending()+0x235) [0x55cabb5b58c5]
 7: (Context::complete(int)+0xd) [0x55cabb4652ad]
 8: (CommonSafeTimer<std::mutex>::timer_thread()+0x13f) [0x7f0061fbe53f]
 9: (CommonSafeTimerThread<std::mutex>::entry()+0x11) [0x7f0061fbf861]
 10: /lib64/libpthread.so.0(+0x81ca) [0x7f005de081ca]
2023-09-23T18:39:35.784 INFO:scrape:1 jobs: ['7401568']
2023-09-23T18:39:35.784 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/xfstests-dev}']
2023-09-23T18:39:35.785 INFO:scrape:
2023-09-23T18:39:35.785 INFO:scrape:Failure: "2023-09-23T06:18:08.947161+0000 mon.a (mon.0) 2777 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering (PG_AVAILABILITY)" in cluster log
2023-09-23T18:39:35.785 INFO:scrape:2 jobs: ['7401555', '7401492']
2023-09-23T18:39:35.785 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}']
2023-09-23T18:39:35.785 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'centos_latest', 'clients/mirror', 'cluster/1-node', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'debug', 'distro/{centos_latest}', 'fs/multifs/{begin/{0-install', 'fs/valgrind/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mirror/{cephfs-mirror/one-per-cluster', 'mon', 'mon-debug}', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-bitmap', 'osd}', 'overrides/ignorelist_health', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/failover}', 'tasks/mirror}}']
2023-09-23T18:39:35.785 INFO:scrape:
2023-09-23T18:39:35.785 INFO:scrape:Timeout 3h running clone.client.1/qa/workunits/suites/fsstress.sh
2023-09-23T18:39:35.786 INFO:scrape:1 jobs: ['7401640']
2023-09-23T18:39:35.786 INFO:scrape:suites: ['1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fail_fs/no', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/quincy']
2023-09-23T18:39:35.786 INFO:scrape:
2023-09-23T18:39:35.786 INFO:scrape:Valgrind: mon (InvalidRead, Leak_PossiblyLost, Leak_StillReachable)
2023-09-23T18:39:35.786 INFO:scrape:1 jobs: ['7401583']
2023-09-23T18:39:35.786 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'fs/verify/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'overrides}}', 'ranks/1', 'session_timeout}', 'tasks/fsstress', 'validater/valgrind}']
2023-09-23T18:39:35.786 INFO:scrape:
2023-09-23T18:39:35.786 INFO:scrape:Dead: None
2023-09-23T18:39:35.787 INFO:scrape:6 jobs: ['7401443', '7401477', '7401562', '7401464', '7401543', '7401556']
2023-09-23T18:39:35.787 INFO:scrape:
2023-09-23T18:39:35.787 INFO:scrape:Failure: error during scrub thrashing: rank damage found: {'backtrace'}
2023-09-23T18:39:35.787 INFO:scrape:2 jobs: ['7401618', '7401639']
2023-09-23T18:39:35.787 INFO:scrape:suites intersection: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'ranks/multi/{export-check', 'replication/default}', 'rhel_8}', 'session_timeout}', 'standby-replay', 'wsync/yes}']
2023-09-23T18:39:35.787 INFO:scrape:suites union: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/yes', '5-workunit/fs/misc}}', '5-workunit/postgres}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'ms_mode/secure', 'n/3', 'n/5', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'ranks/multi/{export-check', 'replication/default}', 'rhel_8}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-no-extra-options}', 'tasks/{0-subvolume/{with-quota}', 'wsync/yes}']
2023-09-23T18:39:35.787 INFO:scrape:
2023-09-23T18:39:35.787 INFO:scrape:Dead: b'2023-09-23T17:11:09.762 DEBUG:teuthology.exit:Finished running handlers'
2023-09-23T18:39:35.788 INFO:scrape:2 jobs: ['7401532', '7401476']
2023-09-23T18:39:35.788 INFO:scrape:suites intersection: ['1-ranks/2', '1-volume/{0-create', '2-client', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fail_fs/yes', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/quincy']
2023-09-23T18:39:35.788 INFO:scrape:suites union: ['1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fail_fs/yes', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/quincy']
2023-09-23T18:39:35.788 INFO:scrape:
2023-09-23T18:39:35.788 INFO:scrape:Failure: Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)
2023-09-23T18:39:35.788 INFO:scrape:1 jobs: ['7401456']
2023-09-23T18:39:35.788 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/data-scan}']
2023-09-23T18:39:35.788 INFO:scrape:
2023-09-23T18:39:35.789 INFO:scrape:Failure: Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS)
2023-09-23T18:39:35.789 INFO:scrape:1 jobs: ['7401487']
2023-09-23T18:39:35.789 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'cluster/{1-node}', 'fs/nfs/{begin/{0-install', 'objectstore/bluestore-bitmap', 'overrides/ignorelist_health', 'supported-random-distros$/{ubuntu_latest}', 'tasks/nfs}']
2023-09-23T18:39:35.789 INFO:scrape:
2023-09-23T18:39:35.789 INFO:scrape:Failure: Command failed (workunit test suites/dbench.sh) on smithi005 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=862663913a74a99e1c52d50e6a7e59dd6705749a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh'
2023-09-23T18:39:35.789 INFO:scrape:1 jobs: ['7401504']
2023-09-23T18:39:35.789 INFO:scrape:suites: ['1-cephadm', '1-check-counter', '2-logrotate}', '2-scrub/yes', '3-snaps/yes', '4-flush/no', '5-workunit/suites/dbench}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-rhel_8', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/5', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'ranks/multi/{export-check', 'replication/default}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-quota}', 'wsync/no}']
2023-09-23T18:39:35.789 INFO:scrape:
2023-09-23T18:39:35.790 INFO:scrape:Failure: Test failure: test_idem_unaffected_root_squash (tasks.cephfs.test_admin.TestFsAuthorizeUpdate)
2023-09-23T18:39:35.790 INFO:scrape:1 jobs: ['7401580']
2023-09-23T18:39:35.790 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/admin}']
2023-09-23T18:39:35.790 INFO:scrape:
2023-09-23T18:39:35.799 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi
=================================================================
info:    http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/
failed:  10
dead:    9
running: 0
waiting: 0
queued:  0
passed:  182


Fail
=================================================================
[7401456]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan}
-----------------------------------------------------------------
time:   00:27:23
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456/
sentry: https://sentry.ceph.com/organizations/ceph/?query=def5a513a6ad4cbf831d15971c7dc327

    Test failure: test_fragmented_injection
    (tasks.cephfs.test_data_scan.TestDataScan)

[7401487]  fs/nfs/{begin/{0-install 1-ceph 2-logrotate} cluster/{1-node} objectstore/bluestore-bitmap overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs}
-----------------------------------------------------------------
time:   00:18:13
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401487/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401487/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a85664d198284f0d9953314eac3c6e4d

    Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS)

[7401492]  fs/valgrind/{begin/{0-install 1-ceph 2-logrotate} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/ignorelist_health tasks/mirror}}
-----------------------------------------------------------------
time:   02:27:44
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401492/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401492/

    "2023-09-23T04:39:06.080013+0000 mon.a (mon.0) 2170 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in
    cluster log

[7401504]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}}
-----------------------------------------------------------------
time:   01:07:58
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401504/
sentry: https://sentry.ceph.com/organizations/ceph/?query=be869081a06548ce91c468e53c6dbbe4

    Command failed (workunit test suites/dbench.sh) on smithi005 with status 1:
    'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=862663913a74a99e1c52d50e6a7e59dd6705749a
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh'

[7401555]  fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
-----------------------------------------------------------------
time:   01:18:18
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401555/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401555/

    "2023-09-23T06:18:08.947161+0000 mon.a (mon.0) 2777 : cluster [WRN] Health
    check failed: Reduced data availability: 2 pgs inactive, 3 pgs peering
    (PG_AVAILABILITY)" in cluster log

[7401580]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/admin}
-----------------------------------------------------------------
time:   00:44:06
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401580/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401580/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a370e65c19ff472da8bd1f7c64ca1f1c

    Test failure: test_idem_unaffected_root_squash
    (tasks.cephfs.test_admin.TestFsAuthorizeUpdate)

[7401583]  fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/fsstress validater/valgrind}
-----------------------------------------------------------------
time:   00:30:48
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401583/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401583/
sentry: https://sentry.ceph.com/organizations/ceph/?query=704b2d1c30134bd18b13c95c9ea59f1e

    saw valgrind issues

[7401618]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}}
-----------------------------------------------------------------
time:   00:50:55
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401618/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401618/

    error during scrub thrashing: rank damage found: {'backtrace'}

[7401639]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}}
-----------------------------------------------------------------
time:   00:49:12
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401639/

    error during scrub thrashing: reached maximum tries (31) after waiting for
    900 seconds

[7401640]  fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   04:42:45
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401640/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401640/
sentry: https://sentry.ceph.com/organizations/ceph/?query=931424ac67024eda964845c8ccd99b6e

    Command failed (workunit test suites/fsstress.sh) on smithi064 with status
    124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd --
    /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=862663913a74a99e1c52d50e6a7e59dd6705749a
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1
    CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh'



Dead
=================================================================
[7401443]  fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401443/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401443/

    Error reimaging machines: reached maximum tries (101) after waiting for 600
    seconds

[7401464]  fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/kernel_untar_build}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401464/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401464/

    Error reimaging machines: reached maximum tries (101) after waiting for 600
    seconds

[7401476]  fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401476/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401476/

    hit max job timeout

[7401477]  fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401477/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401477/

    Error reimaging machines: Timeout opening channel.

[7401532]  fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401532/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401532/

    hit max job timeout

[7401543]  fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401543/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401543/

    Error reimaging machines: reached maximum tries (101) after waiting for 600
    seconds

[7401556]  fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401556/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401556/

    Error reimaging machines: reached maximum tries (101) after waiting for 600
    seconds

[7401562]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/dir-max-entries}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401562/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401562/

    Error reimaging machines: Timeout opening channel.

[7401568]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401568/
log:    http://qa-proxy.ceph.com/teuthology/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401568/

    hit max job timeout



Pass
=================================================================
[7401440] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:31:49
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401440/

[7401441] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}}
time:   00:25:12
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401441/

[7401442] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-readahead}
time:   00:16:07
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401442/

[7401444] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/direct_io}}
time:   00:26:31
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401444/

[7401445] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}}
time:   00:22:21
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401445/

[7401446] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery}
time:   00:53:04
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401446/

[7401447] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:37:11
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401447/

[7401448] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest}
time:   00:27:18
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401448/

[7401449] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:33:42
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401449/

[7401450] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   01:06:17
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401450/

[7401451] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}}
time:   00:19:31
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401451/

[7401452] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage}
time:   00:29:16
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401452/

[7401453] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/misc}}
time:   01:16:10
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401453/

[7401454] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}}
time:   00:16:54
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401454/

[7401455] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:27:52
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401455/

[7401457] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:23:58
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401457/

[7401458] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:19:23
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401458/

[7401459] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports}
time:   00:50:12
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401459/

[7401460] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:41:47
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401460/

[7401461] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}}
time:   01:43:46
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401461/

[7401462] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc}
time:   00:25:26
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401462/

[7401463] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/lockdep}
time:   00:47:47
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401463/

[7401465] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:37:18
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401465/

[7401466] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}
time:   00:23:10
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401466/

[7401467] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}}
time:   00:57:43
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401467/

[7401468] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}}
time:   00:13:55
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401468/

[7401469] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/fragment}
time:   00:24:54
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401469/

[7401470] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}}
time:   00:57:10
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401470/

[7401471] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}}
time:   00:28:56
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401471/

[7401472] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/journal-repair}
time:   00:21:30
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401472/

[7401473] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/postgres}}
time:   00:57:37
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401473/

[7401474] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:21:26
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401474/

[7401475] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush}
time:   00:15:36
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401475/

[7401478] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:20:19
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401478/

[7401479] fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate} centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/trim-i24137}
time:   00:16:49
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401479/

[7401480] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/single-container-host}
time:   00:50:55
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401480/

[7401481] fs/full/{begin/{0-install 1-ceph 2-logrotate} clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mgr-osd-full}
time:   00:24:44
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401481/

[7401482] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client}
time:   00:16:45
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401482/

[7401483] fs/mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{centos_8} tasks/mirror}
time:   02:15:31
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401483/

[7401484] fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distro$/{ubuntu_20.04} workloads/cephfs-mirror-ha-workunit}
time:   00:34:49
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401484/

[7401485] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests}
time:   01:00:29
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401485/

[7401486] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
time:   01:16:04
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401486/

[7401488] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:19:02
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401488/

[7401489] fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell}
time:   00:49:44
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401489/

[7401490] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   01:00:54
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401490/

[7401491] fs/top/{begin/{0-install 1-ceph 2-logrotate} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/ignorelist_health supported-random-distros$/{centos_8} tasks/fstop}
time:   00:19:23
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401491/

[7401493] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}
time:   00:29:50
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401493/

[7401494] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}}
time:   00:38:29
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401494/

[7401495] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:31:57
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401495/

[7401496] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:18:50
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401496/

[7401497] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds_creation_retry}
time:   00:18:25
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401497/

[7401498] fs/upgrade/nofs/{bluestore-bitmap centos_8 conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-pacific 1-upgrade}}
time:   00:17:19
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401498/

[7401499] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics}
time:   00:38:56
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401499/

[7401500] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}}
time:   00:20:05
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401500/

[7401501] fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/new_ops/{0-clients/fuse-upgrade 1-client-sanity}}}
time:   00:17:25
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401501/

[7401502] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/{0-client 1-tests/fscrypt-iozone}}
time:   02:15:41
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401502/

[7401503] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}
time:   00:22:19
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401503/

[7401505] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc}
time:   00:21:49
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401505/

[7401506] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:38:44
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401506/

[7401507] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc}
time:   00:51:36
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401507/

[7401508] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:21:04
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401508/

[7401509] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/fsstress validater/valgrind}
time:   01:42:07
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401509/

[7401510] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable}
time:   00:18:14
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401510/

[7401511] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:37:59
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401511/

[7401512] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}}
time:   00:22:58
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401512/

[7401513] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}}
time:   00:40:11
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401513/

[7401514] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}
time:   00:15:09
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401514/

[7401515] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:19:42
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401515/

[7401516] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:17:32
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401516/

[7401517] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:25:45
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401517/

[7401518] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}}
time:   00:17:25
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401518/

[7401519] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}
time:   00:21:40
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401519/

[7401520] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}}
time:   00:45:06
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401520/

[7401521] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/recovery-fs}
time:   00:13:56
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401521/

[7401522] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file}
time:   00:16:10
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401522/

[7401523] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:27:15
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401523/

[7401524] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:46:50
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401524/

[7401525] fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/fsstress}}}
time:   00:22:59
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401525/

[7401526] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{no-subvolume} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}}
time:   00:41:39
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401526/

[7401527] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub}
time:   00:32:35
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401527/

[7401528] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}}
time:   01:21:48
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401528/

[7401529] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}}
time:   00:25:34
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401529/

[7401530] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}
time:   00:42:31
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401530/

[7401531] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap}
time:   00:22:51
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401531/

[7401533] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}}
time:   00:18:07
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401533/

[7401534] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsstress}}
time:   00:27:23
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401534/

[7401535] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
time:   01:51:02
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401535/

[7401536] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}}
time:   00:50:39
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401536/

[7401537] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:32
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401537/

[7401538] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_20.04} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/ino_release_cb}
time:   00:15:29
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401538/

[7401539] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:18:07
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401539/

[7401540] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir}
time:   00:18:34
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401540/

[7401541] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:44:14
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401541/

[7401542] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc}
time:   00:39:23
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401542/

[7401544] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}}
time:   00:41:19
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401544/

[7401545] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:31:53
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401545/

[7401546] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots}
time:   00:38:52
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401546/

[7401547] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/lockdep}
time:   00:52:35
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401547/

[7401548] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}}
time:   00:17:29
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401548/

[7401549] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:36:46
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401549/

[7401550] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
time:   00:31:17
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401550/

[7401551] fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/kernel_untar_build}}}
time:   01:34:10
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401551/

[7401552] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration}
time:   00:24:12
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401552/

[7401553] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:23:55
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401553/

[7401554] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest}
time:   00:17:05
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401554/

[7401557] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}
time:   01:54:39
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401557/

[7401558] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}}
time:   00:25:52
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401558/

[7401559] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:26:16
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401559/

[7401560] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay}
time:   00:17:41
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401560/

[7401561] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}}
time:   00:19:44
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401561/

[7401563] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:17:44
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401563/

[7401564] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:18:56
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401564/

[7401565] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}}
time:   00:21:30
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401565/

[7401566] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/test_o_trunc}}
time:   00:42:55
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401566/

[7401567] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:37:52
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401567/

[7401569] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:18:38
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401569/

[7401570] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota}
time:   00:21:36
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401570/

[7401571] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}}
time:   00:25:14
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401571/

[7401572] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}}
time:   00:36:29
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401572/

[7401573] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}}
time:   00:18:37
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401573/

[7401574] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/acls}
time:   00:21:03
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401574/

[7401575] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:40:18
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401575/

[7401576] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc}
time:   00:23:33
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401576/

[7401577] fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/dbench}}}
time:   00:57:07
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401577/

[7401578] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn} tasks/{0-client 1-tests/fscrypt-dbench}}
time:   01:16:00
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401578/

[7401579] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}}
time:   00:58:49
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401579/

[7401581] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:18:03
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401581/

[7401582] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:36:53
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401582/

[7401584] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}
time:   00:15:09
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401584/

[7401585] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}}
time:   00:40:19
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401585/

[7401586] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}}
time:   00:50:22
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401586/

[7401587] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:17:44
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401587/

[7401588] fs/cephadm/renamevolume/{0-start 1-rename distro/single-container-host overrides/ignorelist_health}
time:   00:23:52
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401588/

[7401589] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs/{frag test}}
time:   00:34:24
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401589/

[7401590] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests}
time:   01:01:03
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401590/

[7401591] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:46:25
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401591/

[7401592] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:19:00
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401592/

[7401593] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   01:49:20
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401593/

[7401594] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/asok_dump_tree}
time:   00:23:18
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401594/

[7401595] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:32:05
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401595/

[7401596] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}}
time:   00:47:15
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401596/

[7401597] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/auto-repair}
time:   00:15:09
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401597/

[7401598] fs/upgrade/nofs/{bluestore-bitmap centos_8 conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-pacific 1-upgrade}}
time:   00:16:46
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401598/

[7401599] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/pjd}}
time:   00:26:42
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401599/

[7401600] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}}
time:   00:50:03
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401600/

[7401601] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/backtrace}
time:   00:15:42
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401601/

[7401602] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}}
time:   00:18:11
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401602/

[7401603] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cap-flush}
time:   00:15:49
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401603/

[7401604] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:38:13
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401604/

[7401605] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}}
time:   00:15:30
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401605/

[7401606] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/direct_io}}
time:   00:37:47
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401606/

[7401607] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}}
time:   00:17:03
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401607/

[7401608] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits}
time:   00:22:43
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401608/

[7401609] fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/iozone}}}
time:   00:26:56
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401609/

[7401610] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:19:54
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401610/

[7401611] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:25:46
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401611/

[7401612] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-readahead}
time:   00:18:34
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401612/

[7401613] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:27:59
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401613/

[7401614] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}}
time:   00:19:35
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401614/

[7401615] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
time:   00:30:19
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401615/

[7401616] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc}
time:   00:24:16
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401616/

[7401617] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery}
time:   01:03:46
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401617/

[7401619] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}}
time:   01:38:31
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401619/

[7401620] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file}
time:   00:15:57
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401620/

[7401621] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
time:   01:13:44
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401621/

[7401622] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:37:36
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401622/

[7401623] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}}
time:   00:37:34
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401623/

[7401624] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage}
time:   00:25:24
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401624/

[7401625] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/dbench validater/lockdep}
time:   00:50:08
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401625/

[7401626] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}}
time:   00:22:40
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401626/

[7401627] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8 clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}}
time:   00:26:44
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401627/

[7401628] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan}
time:   00:47:07
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401628/

[7401629] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/kernel_untar_build}}
time:   00:48:46
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401629/

[7401630] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:20:30
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401630/

[7401631] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports}
time:   01:03:15
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401631/

[7401632] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}
time:   00:21:52
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401632/

[7401633] fs/upgrade/upgraded_client/{bluestore-bitmap centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/blogbench}}}
time:   00:27:43
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401633/

[7401634] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/pjd}}
time:   00:17:27
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401634/

[7401635] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}
time:   00:17:25
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401635/

[7401636] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:19:13
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401636/

[7401637] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs_python}
time:   00:17:40
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401637/

[7401638] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:19:14
info:   http://pulpito.ceph.com/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401638/