2026-02-24T05:06:51.084 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-24T05:06:51.085 INFO:teuthology.results:subset: '283/1024' 2026-02-24T05:06:51.085 INFO:teuthology.results:seed: '2406' 2026-02-24T09:55:29.534 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-24T09:55:29.540 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2026-02-24T15:24:58.541 INFO:teuthology.results:Tests finished! gathering results... 2026-02-24T15:25:05.765 INFO:scrape:Found 263 jobs 2026-02-24T15:25:41.903 WARNING:scrape:Missing teuthology log /home/teuthworker/mnt/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66922/teuthology.log 2026-02-24T15:25:55.486 INFO:scrape:Found 29 distinct failure reasons 2026-02-24T15:25:55.486 INFO:scrape:Failure: error during thrashing 2026-02-24T15:25:55.486 INFO:scrape:10 jobs: ['67035', '66963', '67089', '66930', '67013', '66958', '67111', '67119', '67041', '67152'] 2026-02-24T15:25:55.487 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'conf/{client', 'frag', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{client-shutdown', 'pg_health', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}'] 2026-02-24T15:25:55.487 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '2-workunit/cfuse_workunit_suites_fsstress}}', '2-workunit/cfuse_workunit_suites_pjd}}', '2-workunit/cfuse_workunit_trivial_sync}}', '2-workunit/ffsb}}', '2-workunit/fs/trivial_sync}}', '2-workunit/iozone}}', '2-workunit/suites/ffsb}}', '2-workunit/suites/fsstress}}', '3-kernel}', 'clusters/1a3s-mds-2c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_latest}', 'distro/{ubuntu_latest}', 'frag', 'fs/thrash/multifs/{begin/{0-install', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'k-stock}', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{client-shutdown', 'overrides/{distro/stock/{centos_9.stream', 'overrides/{distro/testing/k-testing', 'pg_health', 'prefetch_entire_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'prefetch_oft_dirfrags/no', 'prefetch_oft_dirfrags/yes', 'races', 'ranks/1', 'ranks/3', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}'] 2026-02-24T15:25:55.487 INFO:scrape: 2026-02-24T15:25:55.487 INFO:scrape:Failure: 2026-02-24T15:25:55.487 INFO:scrape:7 jobs: ['66955', '67057', '66940', '67165', '67082', '66932', '67033'] 2026-02-24T15:25:55.487 INFO:scrape:suites intersection: ['centos_9.stream', 'conf/{client', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health}'] 2026-02-24T15:25:55.487 INFO:scrape:suites union: ['1-client', '1-mount/mount/fuse', '2-clients/fuse-upgrade', '2-upgrade', '3-client-upgrade', '3-compat_client/no}}', '3-compat_client/yes}}', '3-workload/stress_tests/fsstress}}', '4-compat_client', '5-client-sanity}}', 'centos_9.stream', 'clusters/1-mds-2-client-micro', 'clusters/{1-mds-1-client-micro}', 'conf/{client', 'fs/upgrade/featureful_client/old_client/{bluestore-bitmap', 'fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap', 'fs/upgrade/upgraded_client/{bluestore-bitmap', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'multimds/no', 'multimds/yes', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health}', 'tasks/{0-from/reef', 'tasks/{0-install/reef'] 2026-02-24T15:25:55.487 INFO:scrape: 2026-02-24T15:25:55.487 INFO:scrape:Failure: Test failure: test_acls (tasks.cephfs.test_acls.TestACLs.test_acls) 2026-02-24T15:25:55.487 INFO:scrape:1 jobs: ['66981'] 2026-02-24T15:25:55.487 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{0-use-snaprealm/{global}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/acls}'] 2026-02-24T15:25:55.487 INFO:scrape: 2026-02-24T15:25:55.487 INFO:scrape:Failure: Command failed (workunit test fs/fscrypt_kclient_fuse.sh) on trial106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/fscrypt-fuse && cd -- /home/ubuntu/cephtest/mnt.0/fscrypt-fuse && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 4h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt_kclient_fuse.sh fuse' 2026-02-24T15:25:55.487 INFO:scrape:1 jobs: ['67049'] 2026-02-24T15:25:55.487 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{centos_latest}', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'k-stock}', 'kclient-overrides/{distro/stock/{centos_9.stream', 'mds', 'mgr', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-ec-root', 'osd-asserts', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/fscrypt-kbuild}'] 2026-02-24T15:25:55.487 INFO:scrape: 2026-02-24T15:25:55.487 INFO:scrape:Failure: "2026-02-24T09:39:12.264944+0000 mon.a (mon.0) 1472 : cluster [WRN] Health check failed: 0 slow ops, oldest one blocked for 33 sec, mon.a has slow ops (SLOW_OPS)" in cluster log 2026-02-24T15:25:55.487 INFO:scrape:2 jobs: ['67118', '67145'] 2026-02-24T15:25:55.487 INFO:scrape:suites intersection: ['0-use-snaprealm/{subvol}', '1-cephadm', '1-check-counter', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'pg_health', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}'] 2026-02-24T15:25:55.487 INFO:scrape:suites union: ['0-subvolume/{with-namespace-isolated-and-quota}', '0-subvolume/{with-namespace-isolated}', '0-use-snaprealm/{subvol}', '1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/no', '2-scrub/yes', '3-kernel}', '3-snaps/no', '3-snaps/yes', '4-flush/no', '4-flush/yes', '5-quiesce/no', '5-quiesce/with-quiesce', '6-workunit/fs/norstats}}', '6-workunit/suites/pjd}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'k-stock}', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{base/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'n/3', 'n/5', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{centos_9.stream', 'pg_health', 'ranks/multi/{balancer/automatic', 'ranks/multi/{balancer/random', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}', 'wsync/yes}'] 2026-02-24T15:25:55.487 INFO:scrape: 2026-02-24T15:25:55.487 INFO:scrape:Failure: Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.test_mirroring.TestMirroring.test_cephfs_mirror_restart_sync_on_blocklist) 2026-02-24T15:25:55.488 INFO:scrape:1 jobs: ['66925'] 2026-02-24T15:25:55.488 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'centos_latest', 'clients/mirror', 'cluster/1-node', 'debug', 'fs/valgrind/{begin/{0-install', 'mirror/{cephfs-mirror/one-per-cluster', 'mount/fuse', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/mirror}}'] 2026-02-24T15:25:55.488 INFO:scrape: 2026-02-24T15:25:55.488 INFO:scrape:Failure: reached maximum tries (50) after waiting for 300 seconds 2026-02-24T15:25:55.488 INFO:scrape:5 jobs: ['67138', '67071', '67174', '66969', '67107'] 2026-02-24T15:25:55.488 INFO:scrape:suites intersection: ['1-volume/{0-create', '2-client/fuse', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/yes', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}'] 2026-02-24T15:25:55.488 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client/fuse', '3-inline/no', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/yes', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}', 'tasks/{0-from/reef/{reef}', 'tasks/{0-from/reef/{v18.2.0}', 'tasks/{0-from/reef/{v18.2.1}'] 2026-02-24T15:25:55.488 INFO:scrape: 2026-02-24T15:25:55.488 INFO:scrape:Failure: Command failed (workunit test fs/fscrypt.sh) on trial090 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh unlocked dbench' 2026-02-24T15:25:55.488 INFO:scrape:2 jobs: ['67125', '67026'] 2026-02-24T15:25:55.488 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'bluestore-bitmap', 'clusters/1-mds-1-client', 'conf/{client', 'fs/fscrypt/{begin/{0-install', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'osd', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health}'] 2026-02-24T15:25:55.488 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '3-kernel}', 'bluestore-bitmap', 'clusters/1-mds-1-client', 'conf/{client', 'distro/{centos_latest}', 'distro/{ubuntu_latest}', 'fs/fscrypt/{begin/{0-install', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'osd', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health}', 'tasks/{1-tests/fscrypt-dbench}}', 'tasks/{1-tests/fscrypt-iozone}}'] 2026-02-24T15:25:55.488 INFO:scrape: 2026-02-24T15:25:55.488 INFO:scrape:Failure: Test failure: test_fscrypt_clone (tasks.cephfs.test_fscrypt.TestFSCryptVolumes.test_fscrypt_clone) 2026-02-24T15:25:55.488 INFO:scrape:1 jobs: ['67091'] 2026-02-24T15:25:55.488 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'bluestore-bitmap', 'clusters/1-mds-1-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/fscrypt/{begin/{0-install', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'osd', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health}', 'tasks/{1-tests/fscrypt-common}}'] 2026-02-24T15:25:55.488 INFO:scrape: 2026-02-24T15:25:55.488 INFO:scrape:Failure: "2026-02-24T10:00:00.000125+0000 mon.a (mon.0) 1423 : cluster [WRN] [WRN] OSD_SLOW_PING_TIME_BACK: Slow OSD heartbeats on back (longest 1241.087ms)" in cluster log 2026-02-24T15:25:55.488 INFO:scrape:1 jobs: ['67140'] 2026-02-24T15:25:55.488 INFO:scrape:suites: ['0-subvolume/{with-namespace-isolated-and-quota}', '0-use-snaprealm/{subvol}', '1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/yes', '3-kernel}', '3-snaps/no', '4-flush/no', '5-quiesce/no', '6-workunit/suites/iogen}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/3', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'pg_health', 'ranks/multi/{balancer/automatic', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}', 'wsync/no}'] 2026-02-24T15:25:55.488 INFO:scrape: 2026-02-24T15:25:55.488 INFO:scrape:Failure: Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull.test_full_fsync) 2026-02-24T15:25:55.488 INFO:scrape:1 jobs: ['67079'] 2026-02-24T15:25:55.488 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{0-use-snaprealm/{subvol}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/mds-full}'] 2026-02-24T15:25:55.488 INFO:scrape: 2026-02-24T15:25:55.488 INFO:scrape:Failure: Command failed (workunit test fs/misc/fallocate.sh) on trial082 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/fallocate.sh' 2026-02-24T15:25:55.489 INFO:scrape:2 jobs: ['67153', '66931'] 2026-02-24T15:25:55.489 INFO:scrape:suites intersection: ['0-subvolume/{with-no-extra-options}', '0-use-snaprealm/{global}', '1-cephadm', '1-check-counter', '2-logrotate', '3-kernel}', '4-flush/no', '5-quiesce/no', '6-workunit/fs/misc}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/3', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'pg_health', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}', 'wsync/no}'] 2026-02-24T15:25:55.489 INFO:scrape:suites union: ['0-subvolume/{with-no-extra-options}', '0-use-snaprealm/{global}', '1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/no', '2-scrub/yes', '3-kernel}', '3-snaps/no', '3-snaps/yes', '4-flush/no', '5-quiesce/no', '6-workunit/fs/misc}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/3', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'pg_health', 'ranks/multi/{balancer/automatic', 'ranks/multi/{balancer/random', 'replication/always}', 'replication/default}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}', 'wsync/no}'] 2026-02-24T15:25:55.489 INFO:scrape: 2026-02-24T15:25:55.489 INFO:scrape:Failure: Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.TestJournalMigration.test_journal_migration) 2026-02-24T15:25:55.489 INFO:scrape:2 jobs: ['66944', '67155'] 2026-02-24T15:25:55.489 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'no_client_pidfile', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/test_journal_migration}'] 2026-02-24T15:25:55.489 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{0-use-snaprealm/{global}', 'fs/functional/{0-use-snaprealm/{subvol}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/test_journal_migration}'] 2026-02-24T15:25:55.489 INFO:scrape: 2026-02-24T15:25:55.489 INFO:scrape:Failure: Command failed (workunit test fs/snaps/snap-rm-diff.sh) on trial022 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' 2026-02-24T15:25:55.489 INFO:scrape:2 jobs: ['66982', '67166'] 2026-02-24T15:25:55.489 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '2-workunit/cfuse_workunit_snaptests}}', '3-kernel}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'frag', 'fs/thrash/multifs/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'multifs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{client-shutdown', 'pg_health', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}'] 2026-02-24T15:25:55.489 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '2-workunit/cfuse_workunit_snaptests}}', '3-kernel}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{centos_latest}', 'distro/{ubuntu_latest}', 'frag', 'fs/thrash/multifs/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'k-stock}', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{client-shutdown', 'overrides/{distro/stock/{centos_9.stream', 'pg_health', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}'] 2026-02-24T15:25:55.489 INFO:scrape: 2026-02-24T15:25:55.489 INFO:scrape:Failure: valgrind error: Leak_PossiblyLost operator new(unsigned long) allocate allocate 2026-02-24T15:25:55.489 INFO:scrape:3 jobs: ['67042', '67143', '66942'] 2026-02-24T15:25:55.489 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'fs/verify/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'osd}', 'overrides/{ignorelist_health', 'pg_health', 'session_timeout}', 'tasks/fsstress', 'validater/valgrind}'] 2026-02-24T15:25:55.489 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_latest}', 'distro/{ubuntu/{overrides', 'fs/verify/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{ignorelist_health', 'pg_health', 'ranks/1', 'ranks/3', 'ranks/5', 'session_timeout}', 'tasks/fsstress', 'ubuntu_latest}}', 'validater/valgrind}'] 2026-02-24T15:25:55.489 INFO:scrape: 2026-02-24T15:25:55.489 INFO:scrape:Failure: Test failure: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction) 2026-02-24T15:25:55.489 INFO:scrape:1 jobs: ['67097'] 2026-02-24T15:25:55.489 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/volumes/{0-use-snaprealm/{global}', 'test/misc}}'] 2026-02-24T15:25:55.489 INFO:scrape: 2026-02-24T15:25:55.489 INFO:scrape:Dead: b'2026-02-24T14:17:51.669 DEBUG:teuthology.exit:Finished running handlers' 2026-02-24T15:25:55.489 INFO:scrape:2 jobs: ['67016', '67108'] 2026-02-24T15:25:55.490 INFO:scrape:suites intersection: ['conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}', 'pg_health'] 2026-02-24T15:25:55.490 INFO:scrape:suites union: ['0-subvolume/{with-namespace-isolated-and-quota}', '0-use-snaprealm/{subvol}', '1-cephadm', '1-check-counter', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client/fuse', '2-logrotate', '2-scrub/yes', '3-inline/yes', '3-kernel}', '3-snaps/no', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-flush/no', '4-verify}', '5-quiesce/no', '5-upgrade-with-workload', '6-verify}}', '6-workunit/suites/blogbench}}', 'begin/{0-install', 'centos_9.stream', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'fail_fs/yes', 'frag', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/5', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'ranks/multi/{balancer/random', 'replication/default}', 'roles', 'session_timeout}', 'standby-replay', 'syntax}', 'tasks/{0-from/reef/{v18.2.0}', 'tasks/{0-fscrypt/{with-fscrypt-dummy}', 'wsync/no}'] 2026-02-24T15:25:55.490 INFO:scrape: 2026-02-24T15:25:55.490 INFO:scrape:Failure: Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic) 2026-02-24T15:25:55.490 INFO:scrape:1 jobs: ['66970'] 2026-02-24T15:25:55.490 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_latest}', 'fs/functional/{0-use-snaprealm/{subvol}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/xfstests-dev}'] 2026-02-24T15:25:55.490 INFO:scrape: 2026-02-24T15:25:55.490 INFO:scrape:Failure: Command failed (workunit test suites/dbench.sh) on trial006 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2026-02-24T15:25:55.490 INFO:scrape:2 jobs: ['66943', '67110'] 2026-02-24T15:25:55.490 INFO:scrape:suites intersection: ['0-use-snaprealm/{subvol}', '1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/no', '3-kernel}', '3-snaps/yes', '6-workunit/suites/dbench}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'n/5', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'pg_health', 'ranks/multi/{balancer/automatic', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}'] 2026-02-24T15:25:55.490 INFO:scrape:suites union: ['0-subvolume/{with-namespace-isolated-and-quota}', '0-subvolume/{with-namespace-isolated}', '0-use-snaprealm/{subvol}', '1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/no', '3-kernel}', '3-snaps/yes', '4-flush/no', '4-flush/yes', '5-quiesce/no', '5-quiesce/with-quiesce', '6-workunit/suites/dbench}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'k-stock}', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'ms_mode/secure', 'n/5', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{centos_9.stream', 'overrides/{distro/testing/k-testing', 'pg_health', 'ranks/multi/{balancer/automatic', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}', 'wsync/no}', 'wsync/yes}'] 2026-02-24T15:25:55.490 INFO:scrape: 2026-02-24T15:25:55.490 INFO:scrape:Failure: Command failed (workunit test fs/misc/direct_io.py) on trial036 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py' 2026-02-24T15:25:55.490 INFO:scrape:1 jobs: ['67039'] 2026-02-24T15:25:55.490 INFO:scrape:suites: ['0-subvolume/{with-no-extra-options}', '0-use-snaprealm/{global}', '1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/no', '3-kernel}', '3-snaps/yes', '4-flush/yes', '5-quiesce/no', '6-workunit/fs/misc}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'n/5', 'objectstore-ec/bluestore-comp', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'pg_health', 'ranks/multi/{balancer/random', 'replication/default}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}'] 2026-02-24T15:25:55.490 INFO:scrape: 2026-02-24T15:25:55.490 INFO:scrape:Failure: Test failure: test_async_io_fio (tasks.cephfs.test_nfs.TestNFS.test_async_io_fio) 2026-02-24T15:25:55.490 INFO:scrape:1 jobs: ['66920'] 2026-02-24T15:25:55.490 INFO:scrape:suites: ['conf/{client', 'fs/nfs/{cluster/{1-node}', 'ignorelist_health', 'mds', 'mgr', 'mon', 'nfs-version/{4.2}', 'osd}', 'overrides/{ignore_mgr_down', 'pg_health}', 'supported-random-distros$/{ubuntu_latest}', 'tasks/nfs}'] 2026-02-24T15:25:55.490 INFO:scrape: 2026-02-24T15:25:55.490 INFO:scrape:Failure: Test failure: test_random (tasks.cephfs.test_dump_tree.TestDumpTree.test_random) 2026-02-24T15:25:55.490 INFO:scrape:1 jobs: ['66998'] 2026-02-24T15:25:55.490 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_latest}', 'fs/functional/{0-use-snaprealm/{subvol}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/asok_dump_tree}'] 2026-02-24T15:25:55.490 INFO:scrape: 2026-02-24T15:25:55.490 INFO:scrape:Failure: Test failure: test_progress_event_unique_per_process (tasks.cephfs.test_data_scan.TestDataScan.test_progress_event_unique_per_process) 2026-02-24T15:25:55.490 INFO:scrape:1 jobs: ['67044'] 2026-02-24T15:25:55.491 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_latest}', 'fs/functional/{0-use-snaprealm/{subvol}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/data-scan}'] 2026-02-24T15:25:55.491 INFO:scrape: 2026-02-24T15:25:55.491 INFO:scrape:Failure: Command failed (workunit test suites/fsx.sh) on trial025 with status 110: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2026-02-24T15:25:55.491 INFO:scrape:1 jobs: ['67072'] 2026-02-24T15:25:55.491 INFO:scrape:suites: ['0-subvolume/{with-quota}', '0-use-snaprealm/{global}', '1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/no', '3-kernel}', '3-snaps/yes', '4-flush/yes', '5-quiesce/no', '6-workunit/suites/fsx}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'n/3', 'objectstore-ec/bluestore-comp', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'pg_health', 'ranks/multi/{balancer/automatic', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}'] 2026-02-24T15:25:55.491 INFO:scrape: 2026-02-24T15:25:55.491 INFO:scrape:Dead: None 2026-02-24T15:25:55.491 INFO:scrape:1 jobs: ['66922'] 2026-02-24T15:25:55.491 INFO:scrape: 2026-02-24T15:25:55.491 INFO:scrape:Failure: Command failed on trial050 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9946bf96-1166-11f1-936f-d404e6e7d460 -e sha1=5bc9a1ffc19df681248ece8c3c8c7191a571745b -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' 2026-02-24T15:25:55.491 INFO:scrape:3 jobs: ['67154', '67040', '66945'] 2026-02-24T15:25:55.491 INFO:scrape:suites intersection: ['1-volume/{0-create', '2-client/kclient', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/no', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}', 'tasks/{0-from/squid'] 2026-02-24T15:25:55.491 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client/kclient', '3-inline/no', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/no', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}', 'tasks/{0-from/squid'] 2026-02-24T15:25:55.491 INFO:scrape: 2026-02-24T15:25:55.491 INFO:scrape:Timeout 5h running clone.client.0/qa/workunits/kernel_untar_build.sh 2026-02-24T15:25:55.491 INFO:scrape:1 jobs: ['67158'] 2026-02-24T15:25:55.491 INFO:scrape:suites: ['1-mount/mount/fuse', '2-clients/fuse-upgrade', '3-workload/stress_tests/kernel_untar_build}}', 'centos_9.stream', 'clusters/{1-mds-1-client-micro}', 'conf/{client', 'fs/upgrade/upgraded_client/{bluestore-bitmap', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health}', 'tasks/{0-install/reef'] 2026-02-24T15:25:55.491 INFO:scrape: 2026-02-24T15:25:55.491 INFO:scrape:Failure: Command failed (workunit test suites/ffsb.sh) on trial131 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2026-02-24T15:25:55.491 INFO:scrape:1 jobs: ['67002'] 2026-02-24T15:25:55.491 INFO:scrape:suites: ['0-subvolume/{with-quota}', '0-use-snaprealm/{global}', '1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/no', '3-kernel}', '3-snaps/yes', '4-flush/yes', '5-quiesce/no', '6-workunit/suites/ffsb}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/3', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'pg_health', 'ranks/multi/{balancer/automatic', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}', 'wsync/no}'] 2026-02-24T15:25:55.491 INFO:scrape: 2026-02-24T15:25:55.491 INFO:scrape:Assertion: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18e) [0x7f7debd85bf5] 2026-02-24T15:25:55.491 INFO:scrape:ceph version 20.3.0-5537-g5bc9a1ff (5bc9a1ffc19df681248ece8c3c8c7191a571745b) tentacle (dev - RelWithDebInfo) 1: /lib64/libc.so.6(+0x3fc30) [0x7f7deb23fc30] 2: /lib64/libc.so.6(+0x8d03c) [0x7f7deb28d03c] 3: raise() 4: abort() 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18e) [0x7f7debd85bf5] 6: ceph-mds() [0x4de720] 7: (CInode::pop_and_dirty_projected_inode(std::shared_ptr const&, boost::intrusive_ptr const&)+0x1f6) [0x7c5e46] 8: (MutationImpl::apply()+0x93) [0x653333] 9: (Server::_unlink_local_finish(boost::intrusive_ptr const&, CDentry*, CDentry*, unsigned long)+0x122) [0x611e42] 10: (MDSIOContextBase::complete(int)+0x32c) [0x83d50c] 11: (MDSLogContextBase::complete(int)+0x53) [0x83d683] 12: (Finisher::finisher_thread_entry()+0x16d) [0x7f7debe955cd] 13: /lib64/libc.so.6(+0x8b2fa) [0x7f7deb28b2fa] 14: /lib64/libc.so.6(+0x110400) [0x7f7deb310400] 2026-02-24T15:25:55.491 INFO:scrape:1 jobs: ['67062'] 2026-02-24T15:25:55.491 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'bluestore-bitmap', 'clusters/1-mds-1-client', 'conf/{client', 'distro/{centos_latest}', 'fs/fscrypt/{begin/{0-install', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'k-stock}', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'osd', 'osd}', 'overrides/{distro/stock/{centos_9.stream', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health}', 'tasks/{1-tests/fscrypt-pjd}}'] 2026-02-24T15:25:55.491 INFO:scrape: 2026-02-24T15:25:55.502 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial ================================================================= info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/ logs: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/ failed: 56 dead: 3 running: 0 waiting: 0 queued: 0 passed: 204 Fail ================================================================= [66920] fs/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} nfs-version/{4.2} overrides/{ignore_mgr_down ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/nfs} ----------------------------------------------------------------- time: 00:15:04 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66920/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66920/ Test failure: test_async_io_fio (tasks.cephfs.test_nfs.TestNFS.test_async_io_fio) [66925] fs/valgrind/{begin/{0-install 1-ceph 2-logrotate 3-kernel} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/{ignorelist_health pg_health} tasks/mirror}} ----------------------------------------------------------------- time: 01:06:28 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66925/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66925/ Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.te st_mirroring.TestMirroring.test_cephfs_mirror_restart_sync_on_blocklist) [66930] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:15:43 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66930/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66930/ error during thrashing [66931] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/fs/misc}} ----------------------------------------------------------------- time: 01:37:04 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66931/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66931/ Command failed (workunit test fs/misc/fallocate.sh) on trial055 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/fallocate.sh' [66932] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-compat_client/no}} ----------------------------------------------------------------- time: 02:05:14 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66932/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66932/ [66940] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/fsstress}} ----------------------------------------------------------------- time: 02:30:53 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66940/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66940/ [66942] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/1 tasks/fsstress validater/valgrind} ----------------------------------------------------------------- time: 01:02:48 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66942/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66942/ valgrind error: Leak_PossiblyLost operator new(unsigned long) allocate allocate [66943] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} ----------------------------------------------------------------- time: 00:42:57 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66943/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66943/ Command failed (workunit test suites/dbench.sh) on trial006 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [66944] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/test_journal_migration} ----------------------------------------------------------------- time: 00:14:55 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66944/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66944/ Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.T estJournalMigration.test_journal_migration) [66945] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:21:09 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66945/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66945/ Command failed on trial032 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d8032c4-1141-11f1-b2f2-d404e6e7d460 -e sha1=5bc9a1ffc19df681248ece8c3c8c7191a571745b -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [66955] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 02:30:33 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66955/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66955/ [66958] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:10:30 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66958/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66958/ error during thrashing [66963] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}} ----------------------------------------------------------------- time: 00:21:59 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66963/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66963/ error during thrashing [66969] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:26:45 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66969/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66969/ reached maximum tries (50) after waiting for 300 seconds [66970] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/xfstests-dev} ----------------------------------------------------------------- time: 04:34:31 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66970/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66970/ Test failure: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic) [66981] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/acls} ----------------------------------------------------------------- time: 00:12:33 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66981/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66981/ Test failure: test_acls (tasks.cephfs.test_acls.TestACLs.test_acls) [66982] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:15:01 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66982/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66982/ Command failed (workunit test fs/snaps/snap-rm-diff.sh) on trial022 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [66998] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/asok_dump_tree} ----------------------------------------------------------------- time: 00:35:26 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66998/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66998/ Test failure: test_random (tasks.cephfs.test_dump_tree.TestDumpTree.test_random) [67002] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:52:03 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67002/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67002/ Command failed (workunit test suites/ffsb.sh) on trial131 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [67013] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:11:00 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67013/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67013/ error during thrashing [67026] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-iozone}} ----------------------------------------------------------------- time: 00:09:56 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67026/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67026/ Command failed (workunit test fs/fscrypt.sh) on trial162 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh unlocked iozone' [67033] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-compat_client/yes}} ----------------------------------------------------------------- time: 01:48:38 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67033/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67033/ [67035] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:09:25 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67035/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67035/ error during thrashing [67039] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/fs/misc}} ----------------------------------------------------------------- time: 00:26:18 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67039/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67039/ Command failed (workunit test fs/misc/direct_io.py) on trial036 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py' [67040] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:15:54 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67040/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67040/ Command failed on trial084 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fc82ad60-115c-11f1-861d-d404e6e7d460 -e sha1=5bc9a1ffc19df681248ece8c3c8c7191a571745b -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [67041] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:16:15 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67041/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67041/ error during thrashing [67042] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/5 tasks/fsstress validater/valgrind} ----------------------------------------------------------------- time: 00:40:12 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67042/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67042/ valgrind error: Leak_PossiblyLost operator new(unsigned long) allocate allocate [67044] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/data-scan} ----------------------------------------------------------------- time: 00:19:27 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67044/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67044/ Test failure: test_progress_event_unique_per_process (tasks.cephfs.test_dat a_scan.TestDataScan.test_progress_event_unique_per_process) [67049] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/fscrypt-kbuild} ----------------------------------------------------------------- time: 04:15:23 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67049/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67049/ Command failed (workunit test fs/fscrypt_kclient_fuse.sh) on trial106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/fscrypt-fuse && cd -- /home/ubuntu/cephtest/mnt.0/fscrypt-fuse && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 4h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/fscrypt_kclient_fuse.sh fuse' [67057] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-compat_client/no}} ----------------------------------------------------------------- time: 01:51:16 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67057/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67057/ [67062] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-pjd}} ----------------------------------------------------------------- time: 00:31:11 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67062/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67062/ Command failed (workunit test fs/fscrypt.sh) on trial189 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh unlocked pjd' [67071] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:24:53 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67071/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67071/ reached maximum tries (50) after waiting for 300 seconds [67072] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/fsx}} ----------------------------------------------------------------- time: 00:26:56 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67072/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67072/ Command failed (workunit test suites/fsx.sh) on trial025 with status 110: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' [67079] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/mds-full} ----------------------------------------------------------------- time: 00:09:02 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67079/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67079/ Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull.test_full_fsync) [67082] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} ----------------------------------------------------------------- time: 02:28:45 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67082/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67082/ [67089] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:08:56 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67089/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67089/ error during thrashing [67091] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-common}} ----------------------------------------------------------------- time: 00:10:56 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67091/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67091/ Test failure: test_fscrypt_clone (tasks.cephfs.test_fscrypt.TestFSCryptVolumes.test_fscrypt_clone) [67097] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{global} overrides test/misc}} ----------------------------------------------------------------- time: 00:11:22 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67097/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67097/ Test failure: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction) [67107] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:25:06 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67107/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67107/ reached maximum tries (50) after waiting for 300 seconds [67110] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} ----------------------------------------------------------------- time: 00:24:37 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67110/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67110/ Command failed (workunit test suites/dbench.sh) on trial015 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [67111] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:13:19 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67111/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67111/ error during thrashing [67118] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/norstats}} ----------------------------------------------------------------- time: 00:23:18 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67118/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67118/ "2026-02-24T09:39:12.264944+0000 mon.a (mon.0) 1472 : cluster [WRN] Health check failed: 0 slow ops, oldest one blocked for 33 sec, mon.a has slow ops (SLOW_OPS)" in cluster log [67119] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:07:50 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67119/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67119/ error during thrashing [67125] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-dbench}} ----------------------------------------------------------------- time: 00:45:03 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67125/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67125/ Command failed (workunit test fs/fscrypt.sh) on trial090 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh unlocked dbench' [67138] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:25:01 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67138/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67138/ reached maximum tries (50) after waiting for 300 seconds [67140] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/iogen}} ----------------------------------------------------------------- time: 00:28:25 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67140/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67140/ "2026-02-24T10:00:00.000125+0000 mon.a (mon.0) 1423 : cluster [WRN] [WRN] OSD_SLOW_PING_TIME_BACK: Slow OSD heartbeats on back (longest 1241.087ms)" in cluster log [67143] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/3 tasks/fsstress validater/valgrind} ----------------------------------------------------------------- time: 00:41:08 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67143/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67143/ valgrind error: Leak_PossiblyLost operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun [67145] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:15:59 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67145/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67145/ "2026-02-24T09:52:51.339404+0000 mon.a (mon.0) 1396 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log [67152] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}} ----------------------------------------------------------------- time: 00:09:37 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67152/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67152/ error during thrashing [67153] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/fs/misc}} ----------------------------------------------------------------- time: 00:21:58 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67153/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67153/ Command failed (workunit test fs/misc/fallocate.sh) on trial082 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/fallocate.sh' [67154] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:15:46 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67154/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67154/ Command failed on trial050 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9946bf96-1166-11f1-936f-d404e6e7d460 -e sha1=5bc9a1ffc19df681248ece8c3c8c7191a571745b -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [67155] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/test_journal_migration} ----------------------------------------------------------------- time: 00:11:14 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67155/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67155/ Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.T estJournalMigration.test_journal_migration) [67158] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/kernel_untar_build}} ----------------------------------------------------------------- time: 05:34:29 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67158/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67158/ Command failed (workunit test kernel_untar_build.sh) on trial091 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 5h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' [67165] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/reef 1-client 2-upgrade 3-compat_client/yes}} ----------------------------------------------------------------- time: 01:37:04 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67165/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67165/ [67166] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:08:18 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67166/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67166/ Command failed (workunit test fs/snaps/snap-rm-diff.sh) on trial008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5bc9a1ffc19df681248ece8c3c8c7191a571745b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' [67174] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:25:52 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67174/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67174/ reached maximum tries (50) after waiting for 300 seconds Dead ================================================================= [66922] fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/centos_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66922/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66922/ Error reimaging machines: Expected trial163's OS to be centos 9.stream but found ubuntu 22.04 [67016] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67016/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67016/ hit max job timeout [67108] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67108/ log: https://qa-proxy.ceph.com/teuthology/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67108/ killed Pass ================================================================= [66918] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cephfs_misc_tests} time: 01:05:42 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66918/ [66919] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/multifs-auth} time: 00:33:46 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66919/ [66921] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_misc} time: 00:32:48 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66921/ [66923] fs/snaps/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} time: 00:34:31 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66923/ [66924] fs/top/{begin/{0-install 1-ceph 2-logrotate 3-kernel} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/fstop} time: 00:11:42 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66924/ [66926] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{subvol} overrides test/async}} time: 00:59:37 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66926/ [66927] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/direct_io}} time: 00:13:31 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66927/ [66928] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snap_schedule_snapdir} time: 00:15:18 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66928/ [66929] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:15:06 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66929/ [66933] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} time: 00:32:55 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66933/ [66934] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snapshots} time: 00:29:50 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66934/ [66935] fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} kernel no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/reef 1-upgrade}} time: 00:08:13 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66935/ [66936] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/with-quiesce 2-workunit/suites/ffsb}} time: 00:21:28 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66936/ [66937] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/postgres}} time: 00:35:58 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66937/ [66938] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/strays} time: 00:32:40 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66938/ [66939] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} time: 00:39:45 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66939/ [66941] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:33:27 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66941/ [66946] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} time: 00:49:30 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66946/ [66947] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/test_snap_schedule/{overrides/{v1} snap-schedule}} time: 02:09:01 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66947/ [66948] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/fs/norstats}} time: 00:37:55 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66948/ [66949] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/exports 2-workunit/suites/fsstress}} time: 00:31:44 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66949/ [66950] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{subvol} overrides test/basic}} time: 02:00:09 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66950/ [66951] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/truncate_delay} time: 00:15:08 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66951/ [66952] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsstress}} time: 00:35:09 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66952/ [66953] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:15:57 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66953/ [66954] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:15:02 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66954/ [66956] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-dbench}} time: 01:34:36 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66956/ [66957] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/fsx}} time: 00:43:31 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66957/ [66959] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/uninlining} time: 01:03:33 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66959/ [66960] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} time: 00:12:52 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66960/ [66961] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/workunit/dir-max-entries} time: 00:06:39 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66961/ [66962] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/fs/test_o_trunc}} time: 00:35:17 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66962/ [66964] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:41:03 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66964/ [66965] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/ior-shared-file} time: 00:12:32 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66965/ [66966] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/failover} time: 01:20:15 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66966/ [66967] fs/snaps/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} time: 00:18:46 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66967/ [66968] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iogen}} time: 00:21:33 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66968/ [66971] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iozone}} time: 00:25:00 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66971/ [66972] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/iozone}} time: 01:19:32 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66972/ [66973] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:21:01 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66973/ [66974] fs/bugs/multifs_mdsauthcaps/{clusters/1node-cluster-1node-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/{0-install 1-mount/fuse 2-test_admin_squid_client}} time: 01:48:28 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66974/ [66975] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{centos_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client_full} time: 00:23:47 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66975/ [66976] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/workunit/quota} time: 00:08:23 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66976/ [66977] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-compat_client/yes}} time: 00:13:06 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66977/ [66978] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/pjd}} time: 00:14:52 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66978/ [66979] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{subvol} overrides test/clone-progress}} time: 00:31:14 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66979/ [66980] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/direct_io}} time: 00:34:01 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66980/ [66983] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/squid 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-clients/kclient 3-workload/new_ops/{newops}}} time: 00:24:31 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66983/ [66984] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/misc}} time: 01:37:02 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66984/ [66985] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/admin} time: 01:39:11 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66985/ [66986] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:53:29 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66986/ [66987] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress} time: 00:14:21 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66987/ [66988] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_misc} time: 00:23:08 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66988/ [66989] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/kernel_untar_build}} time: 00:50:40 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66989/ [66990] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/pjd}} time: 00:10:50 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66990/ [66991] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-ffsb}} time: 00:45:01 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66991/ [66992] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} time: 00:29:59 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66992/ [66993] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/alternate-pool} time: 00:12:47 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66993/ [66994] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/3 tasks/dbench validater/lockdep} time: 00:43:03 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66994/ [66995] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-compat_client/no}} time: 00:47:07 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66995/ [66996] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/blogbench}} time: 00:31:00 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66996/ [66997] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:11:39 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66997/ [66999] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} time: 00:55:07 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/66999/ [67000] fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} kernel no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/squid 1-upgrade}} time: 00:09:17 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67000/ [67001] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/auto-repair} time: 00:30:26 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67001/ [67003] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/with-quiesce 2-workunit/fs/snaps}} time: 01:41:19 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67003/ [67004] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/blogbench}} time: 00:49:28 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67004/ [67005] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/norstats}} time: 01:12:42 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67005/ [67006] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/backtrace} time: 00:07:46 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67006/ [67007] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:23:35 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67007/ [67008] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/mdtest} time: 00:09:36 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67008/ [67009] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/multifs-auth} time: 00:33:28 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67009/ [67010] fs/snaps/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} time: 00:33:37 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67010/ [67011] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{global} overrides test/clone}} time: 00:47:01 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67011/ [67012] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/fsstress}} time: 00:30:05 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67012/ [67014] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cap-flush} time: 00:07:52 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67014/ [67015] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsx}} time: 01:19:30 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67015/ [67017] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/fsync-tester}} time: 00:26:50 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67017/ [67018] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:09:14 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67018/ [67019] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:09:33 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67019/ [67020] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/exports 2-workunit/suites/ffsb}} time: 00:16:28 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67020/ [67021] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} time: 00:26:14 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67021/ [67022] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:42:53 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67022/ [67023] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:14:57 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67023/ [67024] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-limits} time: 00:15:14 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67024/ [67025] fs/bugs/snapshot_visibility_on_upgrade/{cluster/1-node conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/{0-install 1-test-subvolume-snapshot-default-visibility}} time: 00:07:18 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67025/ [67027] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/ino_release_cb} time: 00:07:16 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67027/ [67028] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/iogen}} time: 00:29:58 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67028/ [67029] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} time: 00:06:48 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67029/ [67030] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/iozone}} time: 00:11:57 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67030/ [67031] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-readahead} time: 00:07:24 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67031/ [67032] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/pjd}} time: 00:25:51 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67032/ [67034] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-recovery} time: 00:40:23 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67034/ [67036] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{subvol} overrides test/finisher_per_module}} time: 00:07:26 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67036/ [67037] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/direct_io}} time: 00:11:26 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67037/ [67038] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/damage} time: 00:17:24 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67038/ [67043] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} time: 00:22:36 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67043/ [67045] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/postgres}} time: 00:28:20 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67045/ [67046] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:08:08 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67046/ [67047] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:08:34 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67047/ [67048] fs/cephadm/renamevolume/{0-start 1-rename conf/{client mds mgr mon osd} distro/single-container-host overrides/{ignorelist_health pg_health}} time: 00:11:02 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67048/ [67050] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cephfs_misc_tests} time: 00:59:35 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67050/ [67051] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/failover} time: 01:40:08 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67051/ [67052] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:09:26 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67052/ [67053] fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} time: 00:31:50 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67053/ [67054] fs/snaps/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} time: 00:39:32 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67054/ [67055] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/exports} time: 00:40:21 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67055/ [67056] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} time: 00:22:01 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67056/ [67058] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} time: 00:48:46 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67058/ [67059] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/forward-scrub} time: 00:13:22 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67059/ [67060] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} time: 00:10:51 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67060/ [67061] fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} kernel no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/reef 1-upgrade}} time: 00:07:40 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67061/ [67063] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} time: 00:17:03 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67063/ [67064] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/fragment} time: 00:29:10 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67064/ [67065] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/fs/norstats}} time: 00:24:52 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67065/ [67066] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{global} overrides test/metrics}} time: 00:10:05 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67066/ [67067] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} time: 00:08:41 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67067/ [67068] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsstress}} time: 00:12:17 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67068/ [67069] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/journal-repair} time: 00:12:50 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67069/ [67070] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:33:59 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67070/ [67073] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/mds-flush} time: 00:07:48 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67073/ [67074] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} time: 00:51:26 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67074/ [67075] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} time: 00:12:01 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67075/ [67076] fs/bugs/multifs_mdsauthcaps/{clusters/1node-cluster-1node-client conf/{client mds mgr mon osd} distro/{centos_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/{0-install 1-mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} 2-test_admin_squid_client}} time: 01:22:09 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67076/ [67077] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/libcephfs/{frag test}} time: 01:21:29 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67077/ [67078] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/fs/test_o_trunc}} time: 00:25:41 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67078/ [67080] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/suites/pjd}} time: 00:10:58 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67080/ [67081] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/iogen}} time: 00:22:15 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67081/ [67083] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress} time: 00:09:47 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67083/ [67084] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_misc} time: 00:09:26 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67084/ [67085] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/mds_creation_retry} time: 00:06:00 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67085/ [67086] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/iozone}} time: 00:26:40 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67086/ [67087] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:21:17 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67087/ [67088] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/metrics} time: 00:22:12 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67088/ [67090] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/pjd}} time: 00:12:24 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67090/ [67092] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:33:34 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67092/ [67093] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/ior-shared-file} time: 00:08:58 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67093/ [67094] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/multifs-auth} time: 00:14:01 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67094/ [67095] fs/snaps/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} time: 00:14:44 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67095/ [67096] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/1 tasks/dbench validater/lockdep} time: 00:42:47 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67096/ [67098] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/direct_io}} time: 00:13:42 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67098/ [67099] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/multimds_misc} time: 00:13:02 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67099/ [67100] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/misc}} time: 00:26:57 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67100/ [67101] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-compat_client/yes}} time: 00:12:47 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67101/ [67102] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/openfiletable} time: 00:10:54 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67102/ [67103] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/kernel_untar_build}} time: 02:01:28 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67103/ [67104] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} time: 00:30:23 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67104/ [67105] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:11:19 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67105/ [67106] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/pool-perm} time: 00:08:00 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67106/ [67109] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/quiesce} time: 00:41:16 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67109/ [67112] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:09:00 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67112/ [67113] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:09:16 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67113/ [67114] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/ffsb}} time: 00:33:32 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67114/ [67115] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/quota} time: 00:07:35 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67115/ [67116] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:09:26 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67116/ [67117] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-compat_client/no}} time: 00:14:54 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67117/ [67120] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{global} overrides test/snapshot}} time: 00:37:28 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67120/ [67121] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/recovery-fs} time: 00:07:39 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67121/ [67122] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/fsstress}} time: 00:14:22 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67122/ [67123] fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} kernel no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/squid 1-upgrade}} time: 00:07:19 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67123/ [67124] fs/bugs/multifs_mdsauthcaps/{clusters/1node-cluster-1node-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/{0-install 1-mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-test_admin_squid_client}} time: 01:20:01 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67124/ [67126] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/libcephfs_python} time: 00:11:53 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67126/ [67127] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} time: 00:23:25 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67127/ [67128] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/scrub} time: 00:22:32 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67128/ [67129] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:13:31 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67129/ [67130] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/fsync-tester}} time: 00:15:37 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67130/ [67131] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/sessionmap} time: 00:09:43 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67131/ [67132] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} time: 00:25:28 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67132/ [67133] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:17:14 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67133/ [67134] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/mdtest} time: 00:09:45 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67134/ [67135] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/failover} time: 01:28:02 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67135/ [67136] fs/snaps/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} time: 00:23:49 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67136/ [67137] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:18:59 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67137/ [67139] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snap_schedule_snapdir} time: 00:07:45 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67139/ [67141] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} time: 00:23:53 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67141/ [67142] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:08:38 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67142/ [67144] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snapshots} time: 00:26:12 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67144/ [67146] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress} time: 00:10:12 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67146/ [67147] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_misc} time: 00:09:35 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67147/ [67148] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:14:53 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67148/ [67149] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/strays} time: 00:21:04 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67149/ [67150] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{subvol} overrides test/subvolume-snapshots-visibility}} time: 00:23:16 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67150/ [67151] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/direct_io}} time: 00:22:46 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67151/ [67156] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} time: 00:25:52 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67156/ [67157] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/with-quiesce 2-workunit/suites/iozone}} time: 00:09:28 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67157/ [67159] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/test_snap_schedule/{overrides/{v2} snap-schedule}} time: 02:20:12 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67159/ [67160] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-ffsb}} time: 00:52:16 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67160/ [67161] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/postgres}} time: 00:34:04 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67161/ [67162] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:16:56 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67162/ [67163] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} time: 00:29:57 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67163/ [67164] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/truncate_delay} time: 00:08:50 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67164/ [67167] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} time: 00:44:08 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67167/ [67168] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/uninlining} time: 01:00:44 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67168/ [67169] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/reef 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/new_ops/{newops}}} time: 00:08:14 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67169/ [67170] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} time: 00:26:23 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67170/ [67171] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/exports 2-workunit/suites/pjd}} time: 00:11:58 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67171/ [67172] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/fs/norstats}} time: 00:15:36 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67172/ [67173] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/workunit/dir-max-entries} time: 00:09:35 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67173/ [67175] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress} time: 00:09:37 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67175/ [67176] fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate 3-kernel} centos_latest clusters/small-cluster conf/{client mds mgr mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/trim-i24137} time: 00:07:55 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67176/ [67177] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench conf/{client mds mgr mon osd} distro/single-container-host overrides/{ignorelist_health pg_health}} time: 00:46:28 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67177/ [67178] fs/full/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-node-4-mds-1-osd conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/mgr-osd-full} time: 00:14:06 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67178/ [67179] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{centos_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client} time: 00:12:34 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67179/ [67180] fs/mirror/{begin/{0-install 1-ceph 2-logrotate 3-kernel} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} time: 01:21:33 info: https://pulpito.ceph.com/vshankar-2026-02-24_05:06:44-fs-wip-vshankar-testing-20260223.155722-testing-default-trial/67180/ 2026-02-24T15:25:55.512 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 33, in main results(args['--archive-dir'], args['--name'], args['--email'], File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 78, in results email_results( File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 95, in email_results smtp = smtplib.SMTP('localhost') ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/smtplib.py", line 255, in __init__ (code, msg) = self.connect(host, port) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/smtplib.py", line 341, in connect self.sock = self._get_socket(host, port, self.timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/smtplib.py", line 312, in _get_socket return socket.create_connection((host, port), timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/socket.py", line 115, in create_connection sock.connect(sa) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 590, in connect self._internal_connect(address) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 634, in _internal_connect raise _SocketError(err, strerror(err)) ConnectionRefusedError: [Errno 111] Connection refused