2023-10-10T18:19:50.859 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-10-10T18:19:50.860 INFO:teuthology.results:subset: '111/12000'
2023-10-10T18:19:50.860 INFO:teuthology.results:seed: '2048'
2023-10-10T21:23:55.550 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-10-10T21:23:55.577 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-10-11T09:23:56.524 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds
2023-10-11T09:24:13.363 INFO:scrape:Found 190 jobs
2023-10-11T09:24:30.044 INFO:scrape:Found 11 distinct failure reasons
2023-10-11T09:24:30.045 INFO:scrape:Failure: "1696969150.205412 osd.0 (osd.0) 3 : cluster [WRN] OSD bench result of 232801.263459 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.0. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log
2023-10-11T09:24:30.045 INFO:scrape:1 jobs: ['7420465']
2023-10-11T09:24:30.046 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/mds-full}']
2023-10-11T09:24:30.046 INFO:scrape:
2023-10-11T09:24:30.047 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=46d1f59b549730ba7f93d940a3d32b94c45a4cf7
2023-10-11T09:24:30.047 INFO:scrape:2 jobs: ['7420457', '7420525']
2023-10-11T09:24:30.047 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'ms-die-on-skipped}', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}']
2023-10-11T09:24:30.048 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'mds', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd-asserts}', 'osd}', 'overrides/{ignorelist_health', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}']
2023-10-11T09:24:30.048 INFO:scrape:
2023-10-11T09:24:30.049 INFO:scrape:Failure: "1696968527.1509778 mon.a (mon.0) 2219 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
2023-10-11T09:24:30.049 INFO:scrape:3 jobs: ['7420428', '7420506', '7420417']
2023-10-11T09:24:30.049 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}']
2023-10-11T09:24:30.050 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'centos_latest', 'cephfs-mirror/one-per-cluster', 'clients/mirror', 'clients/{mirror}', 'cluster/1-node', 'cluster/{1-node}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'debug', 'distro/{rhel_8}', 'fs/functional/{begin/{0-install', 'fs/mirror/{begin/{0-install', 'fs/valgrind/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mirror/{cephfs-mirror/one-per-cluster', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/ignorelist_health', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'overrides/{ignorelist_health}', 'supported-random-distros$/{rhel_8}', 'tasks/mirror}', 'tasks/mirror}}', 'tasks/snap-schedule}']
2023-10-11T09:24:30.050 INFO:scrape:
2023-10-11T09:24:30.051 INFO:scrape:Failure: Command failed on smithi097 with status 8: 'TESTDIR=/home/ubuntu/cephtest bash -s'
2023-10-11T09:24:30.051 INFO:scrape:2 jobs: ['7420544', '7420452']
2023-10-11T09:24:30.051 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{ignorelist_health', 'tasks/ior-shared-file}']
2023-10-11T09:24:30.052 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1-mds-2-client', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/ior-shared-file}']
2023-10-11T09:24:30.052 INFO:scrape:
2023-10-11T09:24:30.053 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi151 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=46d1f59b549730ba7f93d940a3d32b94c45a4cf7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-git-ceph.sh'
2023-10-11T09:24:30.053 INFO:scrape:1 jobs: ['7420406']
2023-10-11T09:24:30.053 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/cfuse_workunit_snaptests}}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/multifs/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{frag', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}']
2023-10-11T09:24:30.054 INFO:scrape:
2023-10-11T09:24:30.054 INFO:scrape:Failure: Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi175 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=46d1f59b549730ba7f93d940a3d32b94c45a4cf7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh'
2023-10-11T09:24:30.055 INFO:scrape:1 jobs: ['7420370']
2023-10-11T09:24:30.055 INFO:scrape:suites: ['1-ceph', '2-logrotate}', '2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'prefetch_dirfrags/no', 'races', 'ranks/5', 'rhel_8}', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}']
2023-10-11T09:24:30.056 INFO:scrape:
2023-10-11T09:24:30.056 INFO:scrape:Failure: Test failure: test_snapshot_remove (tasks.cephfs.test_strays.TestStrays)
2023-10-11T09:24:30.056 INFO:scrape:1 jobs: ['7420522']
2023-10-11T09:24:30.057 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/strays}']
2023-10-11T09:24:30.057 INFO:scrape:
2023-10-11T09:24:30.058 INFO:scrape:Crash: Command failed (workunit test fs/full/subvolume_snapshot_rm.sh) on smithi170 with status 110: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=46d1f59b549730ba7f93d940a3d32b94c45a4cf7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/full/subvolume_snapshot_rm.sh'
2023-10-11T09:24:30.059 INFO:scrape:ceph version 17.2.6-1374-g46d1f59b (46d1f59b549730ba7f93d940a3d32b94c45a4cf7) quincy (stable)
 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe0) [0x562d79272d75]
 2: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x95a) [0x562d7998b6da]
 3: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x2fd) [0x562d7998de6d]
 4: (non-virtual thunk to PrimaryLogPG::queue_transactions(std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<OpRequest>)+0x55) [0x562d7952be35]
 5: (ReplicatedBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr<PGTransaction, std::default_delete<PGTransaction> >&&, eversion_t const&, eversion_t const&, std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&&, std::optional<pg_hit_set_history_t>&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr<OpRequest>)+0x809) [0x562d79780519]
 6: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0xecf) [0x562d79480fff]
 7: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0xf3b) [0x562d794ebabb]
 8: (PrimaryLogPG::do_op(boost::intrusive_ptr<OpRequest>&)+0x35d3) [0x562d794efe13]
 9: (PrimaryLogPG::do_request(boost::intrusive_ptr<OpRequest>&, ThreadPool::TPHandle&)+0xdc9) [0x562d794fae99]
 10: (OSD::dequeue_op(boost::intrusive_ptr<PG>, boost::intrusive_ptr<OpRequest>, ThreadPool::TPHandle&)+0x1c0) [0x562d79345ab0]
 11: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr<PG>&, ThreadPool::TPHandle&)+0x6a) [0x562d796682ca]
 12: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x90d) [0x562d79372f4d]
 13: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x403) [0x562d79b0c983]
 14: (ShardedThreadPool::WorkThreadSharded::entry()+0x14) [0x562d79b0f9c4]
 15: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7fdd232d2609]
2023-10-11T09:24:30.059 INFO:scrape:1 jobs: ['7420414']
2023-10-11T09:24:30.059 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1-node-1-mds-1-osd', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/full/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides', 'overrides/{ignorelist_health', 'tasks/mgr-osd-full}']
2023-10-11T09:24:30.060 INFO:scrape:
2023-10-11T09:24:30.060 INFO:scrape:Dead: b'2023-10-11T09:24:24.297 INFO:journalctl@ceph.mon.smithi098.smithi098.stdout:Oct 11 09:24:23 smithi098 ceph-a1cb2e20-67b5-11ee-8db6-212e2dc638e7-mon.smithi098[112558]: cluster 2023-10-11T09:24:22.358609+0000 mgr.smithi098.ytqqbi (mgr.14162) 23835 : cluster [DBG] pgmap v21096: 65 pgs: 65 active+clean; 196 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail'
2023-10-11T09:24:30.061 INFO:scrape:1 jobs: ['7420548']
2023-10-11T09:24:30.061 INFO:scrape:suites: ['1-ranks/1', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '3-inline/yes', '3-upgrade-with-workload', '4-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'roles', 'syntax}', 'tasks/{0-from/v16.2.4']
2023-10-11T09:24:30.062 INFO:scrape:
2023-10-11T09:24:30.062 INFO:scrape:Failure: Command failed (workunit test libcephfs/test.sh) on smithi049 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=46d1f59b549730ba7f93d940a3d32b94c45a4cf7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh'
2023-10-11T09:24:30.063 INFO:scrape:1 jobs: ['7420512']
2023-10-11T09:24:30.063 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/libcephfs/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/libcephfs/{frag', 'test}}']
2023-10-11T09:24:30.063 INFO:scrape:
2023-10-11T09:24:30.064 INFO:scrape:Failure: Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes)
2023-10-11T09:24:30.064 INFO:scrape:1 jobs: ['7420483']
2023-10-11T09:24:30.064 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/volumes/{overrides', 'test/basic}}']
2023-10-11T09:24:30.065 INFO:scrape:
2023-10-11T09:24:30.074 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/
failed:  14
dead:    0
running: 1
waiting: 0
queued:  0
passed:  175


Fail
=================================================================
[7420370]  fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}}
-----------------------------------------------------------------
time:   00:57:26
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420370/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420370/
sentry: https://sentry.ceph.com/organizations/ceph/?query=85ea02d3d92f4d5d92f33221cbd2708d

    Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi175 with
    status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=46d1f59b549730ba7f93d940a3d32b94c45a4cf7
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh'

[7420406]  fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}}
-----------------------------------------------------------------
time:   00:29:37
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420406/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420406/
sentry: https://sentry.ceph.com/organizations/ceph/?query=768a5d5b4f404f399ea9f5bc5cd2970a

    Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi151
    with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp &&
    cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=46d1f59b549730ba7f93d940a3d32b94c45a4cf7
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1
    CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.1/qa/workunits/fs/snaps/snaptest-git-
    ceph.sh'

[7420414]  fs/full/{begin/{0-install 1-ceph 2-logrotate} clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mgr-osd-full}
-----------------------------------------------------------------
time:   00:35:26
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420414/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420414/
sentry: https://sentry.ceph.com/organizations/ceph/?query=797c0d52fb9f44f7a53f9cad76e0a089

    Command failed (workunit test fs/full/subvolume_snapshot_rm.sh) on
    smithi170 with status 110: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=46d1f59b549730ba7f93d940a3d32b94c45a4cf7
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/fs/full/subvolume_snapshot_rm.sh'

[7420417]  fs/mirror/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distros$/{rhel_8} tasks/mirror}
-----------------------------------------------------------------
time:   01:57:52
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420417/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420417/

    "1696970365.5832682 mon.a (mon.0) 3926 : cluster [WRN] Health check failed:
    Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

[7420428]  fs/valgrind/{begin/{0-install 1-ceph 2-logrotate} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/ignorelist_health tasks/mirror}}
-----------------------------------------------------------------
time:   02:07:06
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420428/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420428/

    "1696968527.1509778 mon.a (mon.0) 2219 : cluster [WRN] Health check failed:
    Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log

[7420452]  fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file}
-----------------------------------------------------------------
time:   00:12:24
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420452/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420452/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bb4c67e30ec14aaf96151a3373dc04d9

    Command failed on smithi088 with status 8: 'TESTDIR=/home/ubuntu/cephtest
    bash -s'

[7420457]  fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
-----------------------------------------------------------------
time:   00:08:53
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420457/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420457/
sentry: https://sentry.ceph.com/organizations/ceph/?query=eb7ecd8459044d7ebbdb1466f7cd594d

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1
    =46d1f59b549730ba7f93d940a3d32b94c45a4cf7

[7420465]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}
-----------------------------------------------------------------
time:   00:22:46
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420465/

    "1696969150.205412 osd.0 (osd.0) 3 : cluster [WRN] OSD bench result of
    232801.263459 IOPS exceeded the threshold limit of 500.000000 IOPS for
    osd.0. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is
    to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio)
    and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log

[7420483]  fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}
-----------------------------------------------------------------
time:   00:23:43
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420483/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b1d56a04f2df451ab790f37b0f0ecb5b

    Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes)

[7420506]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   01:51:12
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420506/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420506/

    "1696972590.160446 mon.a (mon.0) 623 : cluster [WRN] Health check failed:
    Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log

[7420512]  fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs/{frag test}}
-----------------------------------------------------------------
time:   00:17:08
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420512/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420512/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4b3ca030422749d7bbf19694d848b4bc

    Command failed (workunit test libcephfs/test.sh) on smithi049 with status
    127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=46d1f59b549730ba7f93d940a3d32b94c45a4cf7
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh'

[7420522]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays}
-----------------------------------------------------------------
time:   00:31:42
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420522/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420522/
sentry: https://sentry.ceph.com/organizations/ceph/?query=30ea7979bff14c01a8a1b37e46bf0a0d

    Test failure: test_snapshot_remove (tasks.cephfs.test_strays.TestStrays)

[7420525]  fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench}
-----------------------------------------------------------------
time:   00:08:59
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420525/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420525/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3134fdfcf4f84e14b039ddcb5bde7971

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1
    =46d1f59b549730ba7f93d940a3d32b94c45a4cf7

[7420544]  fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file}
-----------------------------------------------------------------
time:   00:11:50
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420544/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420544/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b7c83b62621d48a488e21df3b112fe95

    Command failed on smithi097 with status 8: 'TESTDIR=/home/ubuntu/cephtest
    bash -s'



Running
=================================================================
[7420548] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420548/



Pass
=================================================================
[7420364] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:22:30
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420364/

[7420365] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration}
time:   00:19:52
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420365/

[7420366] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:34:24
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420366/

[7420367] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:15:00
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420367/

[7420368] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs_python}
time:   00:14:34
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420368/

[7420369] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:16:07
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420369/

[7420371] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/lockdep}
time:   00:18:28
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420371/

[7420372] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay}
time:   00:16:46
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420372/

[7420373] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}}
time:   00:26:03
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420373/

[7420374] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:14:46
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420374/

[7420375] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/dir-max-entries}
time:   00:14:43
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420375/

[7420376] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:28:33
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420376/

[7420377] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}}
time:   00:51:34
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420377/

[7420378] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota}
time:   00:14:16
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420378/

[7420379] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:44:58
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420379/

[7420380] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc}
time:   00:17:03
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420380/

[7420381] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/dbench}}}
time:   00:53:36
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420381/

[7420382] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}}
time:   00:37:21
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420382/

[7420383] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/acls}
time:   00:18:33
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420383/

[7420384] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest}
time:   00:14:46
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420384/

[7420385] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
time:   00:54:49
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420385/

[7420386] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:43:36
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420386/

[7420387] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:10
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420387/

[7420388] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}}
time:   00:24:01
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420388/

[7420389] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{no}}
time:   00:40:01
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420389/

[7420390] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/admin}
time:   00:52:13
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420390/

[7420391] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:23:19
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420391/

[7420392] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/fsstress}}
time:   00:21:39
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420392/

[7420393] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:18:28
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420393/

[7420394] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:16:44
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420394/

[7420395] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}
time:   00:13:51
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420395/

[7420396] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/asok_dump_tree}
time:   00:26:38
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420396/

[7420397] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}}
time:   00:19:48
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420397/

[7420398] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}}
time:   00:37:33
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420398/

[7420399] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/auto-repair}
time:   00:12:42
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420399/

[7420400] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:04
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420400/

[7420401] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}}
time:   00:13:01
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420401/

[7420402] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}}
time:   00:15:14
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420402/

[7420403] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/backtrace}
time:   00:13:50
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420403/

[7420404] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}}
time:   00:25:10
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420404/

[7420405] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind}
time:   00:58:54
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420405/

[7420407] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cap-flush}
time:   00:15:13
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420407/

[7420408] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/iozone}}}
time:   00:18:21
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420408/

[7420409] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/iozone} wsync/{no}}
time:   00:27:36
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420409/

[7420410] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/iozone}}
time:   00:21:17
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420410/

[7420411] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:16:25
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420411/

[7420412] fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate} centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/trim-i24137}
time:   00:13:36
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420412/

[7420413] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/single-container-host}
time:   00:45:59
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420413/

[7420415] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits}
time:   00:29:12
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420415/

[7420416] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client}
time:   00:12:37
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420416/

[7420418] fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{ignorelist_health} supported-random-distro$/{ubuntu_latest} workloads/cephfs-mirror-ha-workunit}
time:   00:27:06
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420418/

[7420419] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:40:54
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420419/

[7420420] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests}
time:   00:50:31
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420420/

[7420421] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:28:37
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420421/

[7420422] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:14:17
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420422/

[7420423] fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell}
time:   00:37:09
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420423/

[7420424] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:43:43
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420424/

[7420425] fs/top/{begin/{0-install 1-ceph 2-logrotate} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/ignorelist_health supported-random-distros$/{rhel_8} tasks/fstop}
time:   00:14:29
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420425/

[7420426] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc}
time:   00:22:47
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420426/

[7420427] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:28:38
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420427/

[7420429] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:47
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420429/

[7420430] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-readahead}
time:   00:13:37
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420430/

[7420431] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:19:16
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420431/

[7420432] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/pjd} wsync/{no}}
time:   00:32:13
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420432/

[7420433] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-octopus 1-upgrade}}
time:   00:13:12
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420433/

[7420434] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery}
time:   00:35:23
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420434/

[7420435] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/pjd}}
time:   00:16:25
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420435/

[7420436] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}
time:   00:22:49
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420436/

[7420437] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/new_ops/{0-clients/fuse-upgrade 1-client-sanity}}}
time:   00:13:30
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420437/

[7420438] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage}
time:   00:23:52
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420438/

[7420439] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:18:44
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420439/

[7420440] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan}
time:   00:43:47
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420440/

[7420441] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/direct_io} wsync/{yes}}
time:   00:28:59
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420441/

[7420442] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:22:00
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420442/

[7420443] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports}
time:   00:51:07
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420443/

[7420444] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/snaps}}
time:   01:36:43
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420444/

[7420445] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:16:57
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420445/

[7420446] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:15:53
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420446/

[7420447] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:12
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420447/

[7420448] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}
time:   00:16:49
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420448/

[7420449] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:18:04
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420449/

[7420450] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/fsstress validater/lockdep}
time:   00:18:50
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420450/

[7420451] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/fragment}
time:   00:20:16
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420451/

[7420453] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
time:   00:48:17
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420453/

[7420454] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:39:09
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420454/

[7420455] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/fs/misc} wsync/{no}}
time:   01:13:02
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420455/

[7420456] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/ffsb}}
time:   00:37:35
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420456/

[7420458] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc}
time:   00:48:23
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420458/

[7420459] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}}
time:   00:25:07
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420459/

[7420460] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/journal-repair}
time:   00:23:47
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420460/

[7420461] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}
time:   00:32:43
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420461/

[7420462] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}}
time:   00:42:13
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420462/

[7420463] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush}
time:   00:13:17
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420463/

[7420464] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:45
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420464/

[7420466] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/kernel_untar_build} wsync/{yes}}
time:   00:41:32
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420466/

[7420467] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/fsstress}}
time:   00:20:26
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420467/

[7420468] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:16
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420468/

[7420469] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/ino_release_cb}
time:   00:14:12
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420469/

[7420470] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:17:22
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420470/

[7420471] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds_creation_retry}
time:   00:18:46
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420471/

[7420472] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}}
time:   00:17:23
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420472/

[7420473] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:27:25
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420473/

[7420474] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics}
time:   00:30:11
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420474/

[7420475] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}}
time:   00:15:17
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420475/

[7420476] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc}
time:   00:19:28
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420476/

[7420477] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}}
time:   00:42:27
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420477/

[7420478] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:39
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420478/

[7420479] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable}
time:   00:18:22
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420479/

[7420480] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest}
time:   00:18:03
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420480/

[7420481] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:27:58
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420481/

[7420482] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:37:30
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420482/

[7420484] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}}
time:   01:21:45
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420484/

[7420485] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:21:50
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420485/

[7420486] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind}
time:   00:54:12
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420486/

[7420487] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}
time:   00:14:03
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420487/

[7420488] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/suites/iozone}}
time:   00:21:05
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420488/

[7420489] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:16:46
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420489/

[7420490] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone}
time:   00:42:03
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420490/

[7420491] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:15:14
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420491/

[7420492] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc}
time:   00:45:59
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420492/

[7420493] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/dbench} wsync/{yes}}
time:   01:02:25
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420493/

[7420494] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}
time:   00:14:04
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420494/

[7420495] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:30:47
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420495/

[7420496] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}}
time:   00:19:50
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420496/

[7420497] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/recovery-fs}
time:   00:16:47
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420497/

[7420498] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/blogbench}}}
time:   00:23:44
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420498/

[7420499] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub}
time:   00:28:20
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420499/

[7420500] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/pjd}}
time:   00:16:39
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420500/

[7420501] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/ffsb} wsync/{yes}}
time:   00:41:59
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420501/

[7420502] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}}
time:   00:25:43
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420502/

[7420503] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap}
time:   00:15:58
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420503/

[7420504] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}}
time:   00:18:56
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420504/

[7420505] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}}
time:   01:01:54
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420505/

[7420507] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/snaps}}
time:   01:14:37
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420507/

[7420508] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:32:11
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420508/

[7420509] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:16:40
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420509/

[7420510] fs/cephadm/renamevolume/{0-start 1-rename distro/single-container-host overrides/ignorelist_health}
time:   00:21:22
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420510/

[7420511] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir}
time:   00:11:56
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420511/

[7420513] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests}
time:   00:50:05
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420513/

[7420514] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}
time:   00:47:39
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420514/

[7420515] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd}
time:   00:16:51
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420515/

[7420516] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:50:54
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420516/

[7420517] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay tasks/{0-check-counter workunit/fs/norstats} wsync/{no}}
time:   00:31:00
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420517/

[7420518] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}}
time:   00:27:41
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420518/

[7420519] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}}
time:   00:19:36
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420519/

[7420520] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots}
time:   00:24:47
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420520/

[7420521] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-octopus 1-upgrade}}
time:   00:13:15
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420521/

[7420523] fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/fsstress validater/valgrind}
time:   01:27:42
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420523/

[7420524] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/ffsb}}
time:   00:40:25
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420524/

[7420526] fs/traceless/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc}
time:   00:17:25
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420526/

[7420527] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/nautilus 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-workload/new_ops/{0-clients/kclient 1-client-sanity}}}
time:   00:14:34
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420527/

[7420528] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration}
time:   00:20:34
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420528/

[7420529] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}}
time:   00:27:57
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420529/

[7420530] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}}
time:   00:36:10
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420530/

[7420531] fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/fsstress}}}
time:   00:18:51
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420531/

[7420532] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay}
time:   00:16:39
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420532/

[7420533] fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}}
time:   00:11:39
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420533/

[7420534] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}}
time:   00:31:42
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420534/

[7420535] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/fsstress}}
time:   00:21:49
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420535/

[7420536] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/dir-max-entries}
time:   00:17:15
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420536/

[7420537] fs/32bits/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress}
time:   00:17:38
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420537/

[7420538] fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc}
time:   00:17:08
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420538/

[7420539] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}}
time:   00:22:50
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420539/

[7420540] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay tasks/{0-check-counter workunit/suites/fsx} wsync/{yes}}
time:   00:31:46
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420540/

[7420541] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota}
time:   00:14:30
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420541/

[7420542] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}}
time:   00:17:24
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420542/

[7420543] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/acls}
time:   00:14:34
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420543/

[7420545] fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}
time:   00:23:36
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420545/

[7420546] fs/snaps/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps}
time:   00:30:55
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420546/

[7420547] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}}
time:   00:14:08
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420547/

[7420549] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/admin}
time:   00:57:24
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420549/

[7420550] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}}
time:   00:25:08
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420550/

[7420551] fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}}
time:   00:29:45
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420551/

[7420552] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}}
time:   01:13:07
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420552/

[7420553] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}
time:   00:11:29
info:   http://pulpito.ceph.com/yuriw-2023-10-10_18:16:52-fs-wip-yuri3-testing-2023-10-10-0720-quincy-distro-default-smithi/7420553/