2023-12-15T02:28:51.230 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-15T02:28:51.230 INFO:teuthology.results:subset: '111/12000' 2023-12-15T02:28:51.230 INFO:teuthology.results:seed: '7023' 2023-12-15T06:58:23.512 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-15T06:58:23.539 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-12-15T13:35:58.069 INFO:teuthology.results:Tests finished! gathering results... 2023-12-15T13:36:12.819 INFO:scrape:Found 173 jobs 2023-12-15T13:36:19.048 INFO:scrape:Found 7 distinct failure reasons 2023-12-15T13:36:19.049 INFO:scrape:Failure: "2023-12-15T03:58:40.298002+0000 mon.a (mon.0) 547 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log 2023-12-15T13:36:19.049 INFO:scrape:3 jobs: ['7492031', '7492249', '7492013'] 2023-12-15T13:36:19.049 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'osd}', 'overrides/{ignorelist_health'] 2023-12-15T13:36:19.049 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/metrics}', 'tasks/snap-schedule}', 'tasks/strays}'] 2023-12-15T13:36:19.049 INFO:scrape: 2023-12-15T13:36:19.049 INFO:scrape:Crash: Command failed (workunit test fs/full/subvolume_snapshot_rm.sh) on smithi077 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e2618c881a5556a9d9d57e114850bac9ede2e2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/full/subvolume_snapshot_rm.sh' 2023-12-15T13:36:19.050 INFO:scrape:ceph version 16.2.14-384-g5e2618c8 (5e2618c881a5556a9d9d57e114850bac9ede2e2d) pacific (stable) 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x561440dd8fce] 2: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0x1497) [0x56144142a557] 3: (BlueStore::queue_transactions(boost::intrusive_ptr&, std::vector >&, boost::intrusive_ptr, ThreadPool::TPHandle*)+0x316) [0x561441447806] 4: (non-virtual thunk to PrimaryLogPG::queue_transactions(std::vector >&, boost::intrusive_ptr)+0x58) [0x561441088508] 5: (ReplicatedBackend::submit_transaction(hobject_t const&, object_stat_sum_t const&, eversion_t const&, std::unique_ptr >&&, eversion_t const&, eversion_t const&, std::vector >&&, std::optional&, Context*, unsigned long, osd_reqid_t, boost::intrusive_ptr)+0xcad) [0x561441278f5d] 6: (PrimaryLogPG::issue_repop(PrimaryLogPG::RepGather*, PrimaryLogPG::OpContext*)+0xcf0) [0x561440ff3d00] 7: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0x115d) [0x56144105017d] 8: (PrimaryLogPG::do_op(boost::intrusive_ptr&)+0x35ca) [0x5614410594da] 9: (PrimaryLogPG::do_request(boost::intrusive_ptr&, ThreadPool::TPHandle&)+0xcbb) [0x56144105ff0b] 10: (OSD::dequeue_op(boost::intrusive_ptr, boost::intrusive_ptr, ThreadPool::TPHandle&)+0x309) [0x561440ee36a9] 11: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0x6b) [0x561441145e7b] 12: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xc28) [0x561440f01178] 13: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x5c4) [0x561441586974] 14: (ShardedThreadPool::WorkThreadSharded::entry()+0x14) [0x561441589854] 15: /lib64/libpthread.so.0(+0x81ca) [0x7f69d90d91ca] 2023-12-15T13:36:19.050 INFO:scrape:1 jobs: ['7492047'] 2023-12-15T13:36:19.050 INFO:scrape:suites: ['clusters/1-node-1-mds-1-osd', 'conf/{client', 'distro/{centos_8}', 'fs/full/{begin', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides', 'overrides/{ignorelist_health', 'tasks/mgr-osd-full}'] 2023-12-15T13:36:19.050 INFO:scrape: 2023-12-15T13:36:19.050 INFO:scrape:Failure: Test failure: test_acls (tasks.cephfs.test_acls.TestACLs) 2023-12-15T13:36:19.050 INFO:scrape:1 jobs: ['7492091'] 2023-12-15T13:36:19.050 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/acls}'] 2023-12-15T13:36:19.051 INFO:scrape: 2023-12-15T13:36:19.051 INFO:scrape:Failure: "2023-12-15T04:14:42.577432+0000 mds.a (mds.0) 30 : cluster [WRN] client.4794 isn't responding to mclientcaps(revoke), ino 0x1000000f55e pending pAsLsXsFscr issued pAsLsXsFscrb, sent 300.429639 seconds ago" in cluster log 2023-12-15T13:36:19.051 INFO:scrape:1 jobs: ['7492005'] 2023-12-15T13:36:19.051 INFO:scrape:suites: ['2-workunit/fs/snaps}}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_8}', 'frag', 'fs/thrash/workloads/{begin', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{client-shutdown', 'prefetch_dirfrags/no', 'races', 'ranks/1', 'session_timeout', 'tasks/{1-thrash/osd', 'thrashosds-health}'] 2023-12-15T13:36:19.051 INFO:scrape: 2023-12-15T13:36:19.051 INFO:scrape:Failure: Test failure: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2023-12-15T13:36:19.051 INFO:scrape:1 jobs: ['7492235'] 2023-12-15T13:36:19.052 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/volumes/{begin', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/volumes/{overrides', 'test/misc}}'] 2023-12-15T13:36:19.052 INFO:scrape: 2023-12-15T13:36:19.052 INFO:scrape:Failure: Command failed on smithi142 with status 8: 'TESTDIR=/home/ubuntu/cephtest bash -s' 2023-12-15T13:36:19.052 INFO:scrape:2 jobs: ['7491943', '7492121'] 2023-12-15T13:36:19.052 INFO:scrape:suites intersection: ['conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{ignorelist_health', 'tasks/ior-shared-file}'] 2023-12-15T13:36:19.052 INFO:scrape:suites union: ['clusters/1-mds-2-client', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin', 'ignorelist_wrongly_marked_down}', 'mds', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/ior-shared-file}'] 2023-12-15T13:36:19.052 INFO:scrape: 2023-12-15T13:36:19.052 INFO:scrape:Failure: 2023-12-15T13:36:19.052 INFO:scrape:1 jobs: ['7492017'] 2023-12-15T13:36:19.052 INFO:scrape:suites: ['clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/workload/{begin', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'ms_mode/{secure}', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag', 'ranks/1', 'scrub/yes', 'session_timeout}', 'standby-replay', 'subvolume/{with-no-extra-options}', 'tasks/{0-check-counter', 'workunit/suites/fsync-tester}', 'wsync/{no}}'] 2023-12-15T13:36:19.052 INFO:scrape: 2023-12-15T13:36:19.060 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/ failed: 10 dead: 0 running: 0 waiting: 0 queued: 0 passed: 163 Fail ================================================================= [7491943] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} ----------------------------------------------------------------- time: 00:14:54 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491943/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491943/ sentry: https://sentry.ceph.com/organizations/ceph/?query=61261a690e3547eaad3dfd2e9eb28ec5 Command failed on smithi142 with status 8: 'TESTDIR=/home/ubuntu/cephtest bash -s' [7492005] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 02:05:25 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492005/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492005/ "2023-12-15T04:14:42.577432+0000 mds.a (mds.0) 30 : cluster [WRN] client.4794 isn't responding to mclientcaps(revoke), ino 0x1000000f55e pending pAsLsXsFscr issued pAsLsXsFscrb, sent 300.429639 seconds ago" in cluster log [7492013] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} ----------------------------------------------------------------- time: 01:45:34 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492013/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492013/ "2023-12-15T03:47:07.919323+0000 mon.a (mon.0) 560 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log [7492017] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{secure} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay subvolume/{with-no-extra-options} tasks/{0-check-counter workunit/suites/fsync-tester} wsync/{no}} ----------------------------------------------------------------- time: 00:15:15 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492017/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492017/ sentry: https://sentry.ceph.com/organizations/ceph/?query=500c87f5763747b3806354959bf5e166 [7492031] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/strays} ----------------------------------------------------------------- time: 00:29:01 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492031/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492031/ "2023-12-15T03:58:40.298002+0000 mon.a (mon.0) 547 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log [7492047] fs/full/{begin clusters/1-node-1-mds-1-osd conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mgr-osd-full} ----------------------------------------------------------------- time: 00:31:56 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492047/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492047/ sentry: https://sentry.ceph.com/organizations/ceph/?query=06ab16d6a3b24d24b477b2271533ea55 Command failed (workunit test fs/full/subvolume_snapshot_rm.sh) on smithi077 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e2618c881a5556a9d9d57e114850bac9ede2e2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/full/subvolume_snapshot_rm.sh' [7492091] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/acls} ----------------------------------------------------------------- time: 00:12:34 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492091/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492091/ sentry: https://sentry.ceph.com/organizations/ceph/?query=056c645c9c60458bb5ec3c0cbc79a2bc Test failure: test_acls (tasks.cephfs.test_acls.TestACLs) [7492121] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/ior-shared-file} ----------------------------------------------------------------- time: 00:13:06 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492121/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492121/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1b5d39e0bf4f40c091e114d4c49c3108 Command failed on smithi151 with status 8: 'TESTDIR=/home/ubuntu/cephtest bash -s' [7492235] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} ----------------------------------------------------------------- time: 00:15:37 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492235/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492235/ sentry: https://sentry.ceph.com/organizations/ceph/?query=faee5bf506054faca22bfeb7469230ab Test failure: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) [7492249] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics} ----------------------------------------------------------------- time: 00:26:39 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492249/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492249/ "2023-12-15T07:09:44.591653+0000 mon.a (mon.0) 1834 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log Pass ================================================================= [7491941] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:36:55 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491941/ [7491945] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics} time: 00:29:15 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491945/ [7491947] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} time: 00:14:09 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491947/ [7491949] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay subvolume/{no-subvolume} tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} time: 00:16:37 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491949/ [7491951] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc} time: 00:23:32 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491951/ [7491953] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:22 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491953/ [7491955] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:21:39 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491955/ [7491957] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable} time: 00:16:37 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491957/ [7491959] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 00:23:28 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491959/ [7491961] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:14:05 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491961/ [7491963] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs_python} time: 00:12:48 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491963/ [7491965] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:16:32 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491965/ [7491967] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} time: 00:19:14 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491967/ [7491969] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay subvolume/{with-no-extra-options} tasks/{0-check-counter workunit/suites/fsstress} wsync/{yes}} time: 00:17:17 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491969/ [7491971] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} time: 00:13:32 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491971/ [7491973] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/lockdep} time: 00:20:05 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491973/ [7491975] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:24:47 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491975/ [7491977] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:17:57 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491977/ [7491979] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} time: 00:12:26 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491979/ [7491981] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/recovery-fs} time: 00:16:49 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491981/ [7491983] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:18:50 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491983/ [7491985] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} time: 00:32:16 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491985/ [7491987] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} time: 01:03:56 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491987/ [7491989] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:17:48 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491989/ [7491991] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/fsx} wsync/{no}} time: 00:22:43 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491991/ [7491993] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:00 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491993/ [7491995] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} time: 00:30:24 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491995/ [7491997] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} time: 01:00:12 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491997/ [7491999] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest} time: 00:15:20 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7491999/ [7492001] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} time: 00:19:22 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492001/ [7492003] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/sessionmap} time: 00:18:38 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492003/ [7492007] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:17:07 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492007/ [7492009] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:14:46 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492009/ [7492011] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} time: 00:36:06 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492011/ [7492015] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:25:38 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492015/ [7492019] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:26:53 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492019/ [7492021] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap_schedule_snapdir} time: 00:17:15 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492021/ [7492023] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} time: 00:27:53 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492023/ [7492025] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} time: 00:20:30 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492025/ [7492027] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:32 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492027/ [7492029] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/fuse ms_mode/{legacy} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay subvolume/{with-quota} tasks/{0-check-counter workunit/fs/test_o_trunc} wsync/{yes}} time: 00:25:32 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492029/ [7492033] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind} time: 00:57:24 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492033/ [7492035] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}} time: 00:18:40 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492035/ [7492037] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/test_journal_migration} time: 00:17:27 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492037/ [7492039] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:27:54 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492039/ [7492041] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:16:01 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492041/ [7492043] fs/bugs/client_trim_caps/{begin centos_latest clusters/small-cluster conf/{client mds mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/trim-i24137} time: 00:14:50 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492043/ [7492045] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench distro/centos_latest} time: 00:45:59 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492045/ [7492049] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client} time: 00:15:14 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492049/ [7492051] fs/mirror/{begin cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distros$/{centos_8} tasks/mirror} time: 01:59:00 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492051/ [7492053] fs/mirror-ha/{begin cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} objectstore/bluestore-bitmap overrides/{whitelist_health} supported-random-distro$/{rhel_8} workloads/cephfs-mirror-ha-workunit} time: 02:34:12 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492053/ [7492055] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:31:32 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492055/ [7492057] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:48:24 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492057/ [7492059] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} time: 00:30:03 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492059/ [7492061] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:17:19 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492061/ [7492063] fs/shell/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} time: 00:37:30 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492063/ [7492065] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:53:57 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492065/ [7492067] fs/top/{begin cluster/{1-node} mount/fuse objectstore/bluestore-bitmap overrides/ignorelist_health supported-random-distros$/{centos_8} tasks/fstop} time: 00:14:37 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492067/ [7492069] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:20:27 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492069/ [7492071] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} time: 00:29:53 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492071/ [7492073] fs/valgrind/{begin centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/whitelist_health tasks/mirror}} time: 02:11:18 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492073/ [7492075] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volume-client} time: 00:31:13 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492075/ [7492077] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/truncate_delay} time: 00:18:04 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492077/ [7492079] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay subvolume/{with-no-extra-options} tasks/{0-check-counter workunit/suites/iogen} wsync/{yes}} time: 00:39:57 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492079/ [7492081] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} time: 01:52:51 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492081/ [7492083] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/dir-max-entries} time: 00:17:32 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492083/ [7492085] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota} time: 00:14:22 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492085/ [7492087] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:16:16 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492087/ [7492089] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:51 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492089/ [7492093] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/iozone} wsync/{no}} time: 00:19:25 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492093/ [7492095] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:19:06 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492095/ [7492097] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/admin} time: 00:54:02 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492097/ [7492099] fs/upgrade/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-octopus 1-upgrade}} time: 00:14:39 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492099/ [7492101] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} time: 00:17:03 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492101/ [7492103] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:17:10 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492103/ [7492105] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:15:33 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492105/ [7492107] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} time: 01:31:53 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492107/ [7492109] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} time: 00:15:29 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492109/ [7492111] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay subvolume/{with-quota} tasks/{0-check-counter workunit/suites/pjd} wsync/{yes}} time: 00:20:18 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492111/ [7492113] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} time: 00:23:02 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492113/ [7492115] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/asok_dump_tree} time: 00:21:04 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492115/ [7492117] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/3 tasks/fsstress validater/lockdep} time: 00:20:26 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492117/ [7492119] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:51:12 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492119/ [7492123] fs/upgrade/upgraded_client/from_nautilus/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client-upgrade 2-client-sanity}} time: 00:24:28 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492123/ [7492125] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/auto-repair} time: 00:13:50 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492125/ [7492127] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:15:53 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492127/ [7492129] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} time: 00:54:48 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492129/ [7492131] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:44:46 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492131/ [7492133] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:45:36 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492133/ [7492135] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/backtrace} time: 00:46:12 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492135/ [7492137] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} time: 00:18:32 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492137/ [7492139] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{crc} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay subvolume/{with-quota} tasks/{0-check-counter workunit/direct_io} wsync/{yes}} time: 00:18:36 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492139/ [7492141] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cap-flush} time: 00:15:04 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492141/ [7492143] fs/upgrade/volumes/import-legacy/{bluestore-bitmap clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-verify} ubuntu_18.04} time: 01:11:52 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492143/ [7492145] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 01:49:50 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492145/ [7492147] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:47:40 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492147/ [7492149] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/ino_release_cb} time: 00:13:37 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492149/ [7492151] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:16:48 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492151/ [7492153] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} time: 00:58:29 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492153/ [7492155] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-limits} time: 00:27:20 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492155/ [7492157] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} time: 00:47:01 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492157/ [7492159] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-readahead} time: 00:12:13 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492159/ [7492161] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{legacy} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/yes standby-replay subvolume/{with-no-extra-options} tasks/{0-check-counter workunit/fs/misc} wsync/{no}} time: 00:32:57 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492161/ [7492163] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:49 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492163/ [7492165] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/client-recovery} time: 00:51:50 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492165/ [7492167] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:44:11 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492167/ [7492169] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} time: 00:28:24 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492169/ [7492171] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:36:51 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492171/ [7492173] fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/mdtest} time: 00:20:08 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492173/ [7492175] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} time: 00:14:42 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492175/ [7492177] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/no standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/kernel_untar_build} wsync/{no}} time: 00:37:11 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492177/ [7492179] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/dbench validater/valgrind} time: 00:55:24 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492179/ [7492181] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:26:33 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492181/ [7492183] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} time: 07:42:07 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492183/ [7492185] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} time: 00:20:57 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492185/ [7492187] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:13:24 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492187/ [7492189] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} time: 00:28:03 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492189/ [7492191] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:21:05 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492191/ [7492193] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:40:40 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492193/ [7492195] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:34:38 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492195/ [7492197] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} time: 00:14:46 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492197/ [7492199] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/exports} time: 00:40:36 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492199/ [7492201] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} time: 01:21:10 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492201/ [7492203] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} time: 00:20:56 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492203/ [7492205] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/yes standby-replay subvolume/{with-quota} tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} time: 00:29:22 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492205/ [7492207] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:35:22 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492207/ [7492209] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/fragment} time: 00:22:36 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492209/ [7492211] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:11:52 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492211/ [7492213] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/journal-repair} time: 00:18:52 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492213/ [7492215] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:22:29 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492215/ [7492217] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:25:56 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492217/ [7492219] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse ms_mode/{secure} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 scrub/no standby-replay subvolume/{with-namespace-isolated-and-quota} tasks/{0-check-counter workunit/suites/dbench} wsync/{no}} time: 00:42:34 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492219/ [7492221] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-flush} time: 00:10:08 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492221/ [7492223] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:31:46 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492223/ [7492225] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:17:00 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492225/ [7492227] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/libcephfs/{frag test}} time: 00:20:10 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492227/ [7492229] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:47:18 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492229/ [7492231] fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cephfs_misc_tests} time: 00:48:10 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492231/ [7492233] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} time: 00:16:42 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492233/ [7492237] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} time: 00:27:13 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492237/ [7492239] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:19:16 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492239/ [7492241] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds_creation_retry} time: 00:17:15 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492241/ [7492243] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:38 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492243/ [7492246] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/3 scrub/yes standby-replay subvolume/{with-namespace-isolated} tasks/{0-check-counter workunit/suites/ffsb} wsync/{no}} time: 00:32:18 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492246/ [7492247] fs/verify/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/fsstress validater/valgrind} time: 01:42:41 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492247/ [7492251] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} time: 00:19:12 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492251/ [7492253] fs/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} time: 00:55:04 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492253/ [7492255] fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} time: 00:50:30 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492255/ [7492257] fs/traceless/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:16:38 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492257/ [7492258] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc} time: 00:16:39 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492258/ [7492259] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} time: 00:13:47 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492259/ [7492260] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/yes pg-warn} tasks/{0-nautilus 1-client 2-upgrade 3-compat_client/no}} time: 00:30:39 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492260/ [7492261] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/openfiletable} time: 00:18:41 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492261/ [7492262] fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} ms_mode/{crc} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/5 scrub/no standby-replay subvolume/{with-no-extra-options} tasks/{0-check-counter workunit/fs/norstats} wsync/{yes}} time: 00:18:52 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492262/ [7492263] fs/thrash/workloads/{begin clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} time: 01:46:50 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492263/ [7492264] fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} time: 00:16:54 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492264/ [7492265] fs/permission/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} time: 00:14:34 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492265/ [7492266] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} time: 00:30:57 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492266/ [7492267] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} time: 00:15:07 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492267/ [7492268] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:28:41 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492268/ [7492269] fs/thrash/multifs/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:29:42 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492269/ [7492270] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} time: 00:12:31 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492270/ [7492271] fs/mixed-clients/{begin clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:37:21 info: https://pulpito.ceph.com/yuriw-2023-12-14_22:18:58-fs-wip-yuri5-testing-2023-12-14-1107-pacific-distro-default-smithi/7492271/