2026-02-25T15:29:49.589 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-25T15:29:49.589 INFO:teuthology.results:subset: '111/120000' 2026-02-25T15:29:49.589 INFO:teuthology.results:seed: '6885' 2026-02-25T17:03:47.385 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-25T17:03:47.394 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2026-02-25T22:25:24.273 INFO:teuthology.results:Tests finished! gathering results... 2026-02-25T22:25:29.088 INFO:scrape:Found 193 jobs 2026-02-25T22:25:30.216 WARNING:scrape:Misunderstood line: 2026-02-25T22:25:45.619 WARNING:scrape:Misunderstood line: 2026-02-25T22:25:47.518 WARNING:scrape:Misunderstood line: 2026-02-25T22:26:07.184 INFO:scrape:Found 25 distinct failure reasons 2026-02-25T22:26:07.185 INFO:scrape:Failure: failed to install new kernel version within timeout 2026-02-25T22:26:07.185 INFO:scrape:9 jobs: ['70294', '70329', '70403', '70279', '70253', '70322', '70298', '70390', '70362'] 2026-02-25T22:26:07.185 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-modules}', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'osd}'] 2026-02-25T22:26:07.185 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '2-workunit/cfuse_workunit_snaptests}}', '2-workunit/iozone}}', '2-workunit/suites/iozone}}', '3-modules}', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'distro/{ubuntu_20.04}', 'frag', 'fs/functional/{begin/{0-install', 'fs/thrash/multifs/{begin/{0-install', 'fs/thrash/workloads/{begin/{0-install', 'fs/verify/{begin/{0-install', 'fs/volumes/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'multifs', 'no_client_pidfile', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{client-shutdown', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'overrides}}', 'pg_health', 'pg_health}', 'prefetch_entire_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'prefetch_oft_dirfrags/no', 'prefetch_oft_dirfrags/yes', 'races', 'ranks/3', 'ranks/5', 'session_timeout', 'session_timeout}', 'tasks/asok_dump_tree}', 'tasks/client-recovery}', 'tasks/fsstress', 'tasks/snap_schedule_snapdir}', 'tasks/volumes/{overrides', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/osd', 'test/async}}', 'test/finisher_per_module}}', 'thrashosds-health}', 'validater/lockdep}'] 2026-02-25T22:26:07.185 INFO:scrape: 2026-02-25T22:26:07.185 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=pacific 2026-02-25T22:26:07.185 INFO:scrape:4 jobs: ['70374', '70302', '70241', '70284'] 2026-02-25T22:26:07.185 INFO:scrape:suites intersection: ['1-volume/{0-create', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}'] 2026-02-25T22:26:07.185 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client/fuse', '2-client/kclient', '3-inline/no', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/no', 'fail_fs/yes', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}', 'tasks/{0-from/pacific', 'tasks/{0-from/v16.2.4'] 2026-02-25T22:26:07.185 INFO:scrape: 2026-02-25T22:26:07.185 INFO:scrape:Failure: "2026-02-25T16:45:17.767746+0000 mds.b (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/36804114-ab2d-47f0-8d93-798a4d140c38/postgres/data/pg_wal) see mds.b log and `damage ls` output for details" in cluster log 2026-02-25T22:26:07.185 INFO:scrape:1 jobs: ['70369'] 2026-02-25T22:26:07.185 INFO:scrape:suites: ['1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/yes', '3-modules}', '3-snaps/yes', '4-flush/yes', '5-workunit/postgres}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/secure', 'n/3', 'objectstore-ec/bluestore-comp', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'pg_health', 'ranks/multi/{balancer/random', 'replication/default}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-no-extra-options}', 'wsync/yes}'] 2026-02-25T22:26:07.185 INFO:scrape: 2026-02-25T22:26:07.185 INFO:scrape:Failure: reached maximum tries (50) after waiting for 300 seconds 2026-02-25T22:26:07.185 INFO:scrape:1 jobs: ['70270'] 2026-02-25T22:26:07.185 INFO:scrape:suites: ['1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client/fuse', '3-inline/no', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/yes', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}', 'tasks/{0-from/reef/{v18.2.0}'] 2026-02-25T22:26:07.185 INFO:scrape: 2026-02-25T22:26:07.185 INFO:scrape:Valgrind: client (Leak_StillReachable), mon (Leak_StillReachable) 2026-02-25T22:26:07.185 INFO:scrape:2 jobs: ['70395', '70242'] 2026-02-25T22:26:07.185 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_9.stream}', 'fs/verify/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mon-debug', 'mount/fuse', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'pg_health', 'session_timeout}', 'tasks/dbench', 'validater/valgrind}'] 2026-02-25T22:26:07.185 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_9.stream}', 'fs/verify/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mon-debug', 'mount/fuse', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'pg_health', 'ranks/3', 'ranks/5', 'session_timeout}', 'tasks/dbench', 'validater/valgrind}'] 2026-02-25T22:26:07.186 INFO:scrape: 2026-02-25T22:26:07.186 INFO:scrape:Failure: Test failure: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction) 2026-02-25T22:26:07.186 INFO:scrape:1 jobs: ['70336'] 2026-02-25T22:26:07.186 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_20.04}', 'fs/volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/volumes/{overrides', 'test/misc}}'] 2026-02-25T22:26:07.186 INFO:scrape: 2026-02-25T22:26:07.186 INFO:scrape:Failure: Test failure: test_cephfs_mirror_cancel_sync (tasks.cephfs.test_mirroring.TestMirroring.test_cephfs_mirror_cancel_sync) 2026-02-25T22:26:07.186 INFO:scrape:1 jobs: ['70310'] 2026-02-25T22:26:07.186 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-modules}', 'cephfs-mirror/one-per-cluster', 'clients/{mirror}', 'cluster/{1-node}', 'conf/{client', 'fs/mirror/{begin/{0-install', 'mds', 'mgr', 'mon', 'mount/fuse', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'supported-random-distros$/{ubuntu_latest}', 'tasks/mirror}'] 2026-02-25T22:26:07.186 INFO:scrape: 2026-02-25T22:26:07.186 INFO:scrape:Failure: Test failure: test_fscrypt_dummy_encryption_with_quick_group (tasks.cephfs.test_fscrypt.TestFSCryptXFS.test_fscrypt_dummy_encryption_with_quick_group) 2026-02-25T22:26:07.186 INFO:scrape:1 jobs: ['70247'] 2026-02-25T22:26:07.186 INFO:scrape:suites: ['1-ceph', '1-tests/fscrypt-common}}', '2-logrotate', '3-modules}', 'bluestore-bitmap', 'clusters/1-mds-1-client', 'conf/{client', 'distro/{centos_latest}', 'fs/fscrypt/{begin/{0-install', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health}', 'tasks/{0-client'] 2026-02-25T22:26:07.186 INFO:scrape: 2026-02-25T22:26:07.186 INFO:scrape:Failure: Command failed (workunit test fs/misc/fallocate.sh) on trial026 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0ef62ce8e48de042e1e5e6a70cfa2ee3f1174d28 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/fallocate.sh' 2026-02-25T22:26:07.186 INFO:scrape:1 jobs: ['70350'] 2026-02-25T22:26:07.186 INFO:scrape:suites: ['1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/yes', '3-modules}', '3-snaps/yes', '4-flush/yes', '5-workunit/fs/misc}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'n/5', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'pg_health', 'ranks/multi/{balancer/random', 'replication/default}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-namespace-isolated}', 'wsync/yes}'] 2026-02-25T22:26:07.186 INFO:scrape: 2026-02-25T22:26:07.186 INFO:scrape:Failure: Test failure: test_cephfs_mirror_incremental_sync_with_type_mixup (tasks.cephfs.test_mirroring.TestMirroring.test_cephfs_mirror_incremental_sync_with_type_mixup) 2026-02-25T22:26:07.186 INFO:scrape:1 jobs: ['70321'] 2026-02-25T22:26:07.186 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-modules}', 'centos_latest', 'clients/mirror', 'cluster/1-node', 'debug', 'fs/valgrind/{begin/{0-install', 'mirror/{cephfs-mirror/one-per-cluster', 'mount/fuse', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/mirror}}'] 2026-02-25T22:26:07.186 INFO:scrape: 2026-02-25T22:26:07.186 INFO:scrape:Failure: error during thrashing 2026-02-25T22:26:07.186 INFO:scrape:12 jobs: ['70399', '70246', '70418', '70323', '70335', '70285', '70277', '70382', '70371', '70367', '70342', '70256'] 2026-02-25T22:26:07.186 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-modules}', 'conf/{client', 'frag', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{client-shutdown', 'pg_health', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}'] 2026-02-25T22:26:07.186 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '2-workunit/cfuse_workunit_snaptests}}', '2-workunit/cfuse_workunit_suites_fsstress}}', '2-workunit/cfuse_workunit_suites_pjd}}', '2-workunit/cfuse_workunit_trivial_sync}}', '2-workunit/ffsb}}', '2-workunit/fs/snaps}}', '2-workunit/fs/trivial_sync}}', '2-workunit/suites/ffsb}}', '2-workunit/suites/fsstress}}', '2-workunit/suites/iozone}}', '2-workunit/suites/pjd}}', '3-modules}', 'clusters/1a3s-mds-2c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{centos_latest}', 'distro/{ubuntu_20.04}', 'distro/{ubuntu_latest}', 'frag', 'fs/thrash/multifs/{begin/{0-install', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{client-shutdown', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'pg_health', 'prefetch_entire_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'prefetch_oft_dirfrags/no', 'prefetch_oft_dirfrags/yes', 'races', 'ranks/1', 'ranks/3', 'ranks/5', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}', 'ubuntu_22.04}'] 2026-02-25T22:26:07.186 INFO:scrape: 2026-02-25T22:26:07.187 INFO:scrape:Failure: Test failure: test_with_health_warn_trim (tasks.cephfs.test_admin.TestFSFail.test_with_health_warn_trim) 2026-02-25T22:26:07.187 INFO:scrape:1 jobs: ['70286'] 2026-02-25T22:26:07.187 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/admin}'] 2026-02-25T22:26:07.187 INFO:scrape: 2026-02-25T22:26:07.187 INFO:scrape:Dead: b'2026-02-25T22:25:00.462 DEBUG:teuthology.exit:Finished running handlers' 2026-02-25T22:26:07.187 INFO:scrape:1 jobs: ['70360'] 2026-02-25T22:26:07.187 INFO:scrape:suites: ['1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/no', '3-modules}', '3-snaps/no', '4-flush/no', '5-workunit/kernel_untar_build}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/5', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'pg_health', 'ranks/multi/{balancer/automatic', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-namespace-isolated-and-quota}', 'ubuntu_22.04}', 'wsync/no}'] 2026-02-25T22:26:07.187 INFO:scrape: 2026-02-25T22:26:07.187 INFO:scrape:Failure: "2026-02-25T16:15:11.146345+0000 mds.d (mds.0) 1 : cluster [WRN] client.4628 does not advance its oldest_flush_tid (3923), 100000 completed flushes recorded in session" in cluster log 2026-02-25T22:26:07.187 INFO:scrape:2 jobs: ['70292', '70307'] 2026-02-25T22:26:07.187 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-modules}', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'objectstore/bluestore-bitmap', 'osd}'] 2026-02-25T22:26:07.187 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '2-workunit/ffsb}}', '3-modules}', 'clusters/1-node-4-mds-1-osd', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{centos_latest}', 'distro/{ubuntu_20.04}', 'frag', 'fs/full/{begin/{0-install', 'fs/thrash/multifs/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides', 'overrides/{client-shutdown', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'pg_health', 'pg_health}', 'session_timeout', 'tasks/mgr-osd-full}', 'tasks/{1-thrash/mon', 'thrashosds-health}', 'ubuntu_22.04}'] 2026-02-25T22:26:07.187 INFO:scrape: 2026-02-25T22:26:07.187 INFO:scrape:Failure: Test failure: test_client_evict_with_invalid_id (tasks.cephfs.test_misc.TestSessionClientEvict.test_client_evict_with_invalid_id) 2026-02-25T22:26:07.187 INFO:scrape:2 jobs: ['70410', '70313'] 2026-02-25T22:26:07.187 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/cephfs_misc_tests}'] 2026-02-25T22:26:07.187 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1-mds-3-client', 'conf/{client', 'distros/ubuntu_latest', 'fs/multiclient/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/cephfs_misc_tests}'] 2026-02-25T22:26:07.187 INFO:scrape: 2026-02-25T22:26:07.187 INFO:scrape:Failure: Test failure: test_subvolume_replicated (tasks.cephfs.test_subvolume.TestSubvolumeReplicated.test_subvolume_replicated) 2026-02-25T22:26:07.187 INFO:scrape:1 jobs: ['70269'] 2026-02-25T22:26:07.187 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/volumes/{overrides', 'test/basic}}'] 2026-02-25T22:26:07.187 INFO:scrape: 2026-02-25T22:26:07.187 INFO:scrape:Failure: "grep: /var/log/ceph/64d49534-1264-11f1-9de3-d404e6e7d460/ceph.log: No such file or directory" in cluster log 2026-02-25T22:26:07.187 INFO:scrape:3 jobs: ['70254', '70393', '70339'] 2026-02-25T22:26:07.187 INFO:scrape:suites intersection: ['1-volume/{0-create', '2-allow_standby_replay/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}', 'tasks/{0-from/quincy'] 2026-02-25T22:26:07.187 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/yes', '2-client/fuse', '2-client/kclient', '3-inline/no', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/no', 'fail_fs/yes', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}', 'tasks/{0-from/quincy'] 2026-02-25T22:26:07.187 INFO:scrape: 2026-02-25T22:26:07.188 INFO:scrape:Crash: "grep: /var/log/ceph/ceph.log: No such file or directory" in cluster log 2026-02-25T22:26:07.188 INFO:scrape:ceph version 18.2.7-1050-g0ef62ce8 (0ef62ce8e48de042e1e5e6a70cfa2ee3f1174d28) reef (stable) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x5865420] 2: (tcmalloc::CentralFreeList::FetchFromOneSpans(int, void**, void**)+0x20) [0x56330a0] 3: (tcmalloc::CentralFreeList::FetchFromOneSpansSafe(int, void**, void**)+0x20) [0x5633370] 4: (tcmalloc::CentralFreeList::RemoveRange(void**, void**, int)+0x80) [0x5633430] 5: (tcmalloc::ThreadCache::FetchFromCentralCache(unsigned int, int, void* (*)(unsigned long))+0x76) [0x5636e46] 6: (tcmalloc::allocate_full_cpp_throw_oom(unsigned long)+0x165) [0x5647015] 7: (rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string, std::allocator > const&, std::vector > const&, std::vector >*, rocksdb::DB**, bool, bool)+0x3e4) [0x822a54] 8: (rocksdb::DB::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string, std::allocator > const&, std::vector > const&, std::vector >*, rocksdb::DB**)+0x29) [0x824f79] 9: (RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string, std::allocator > const&)+0x794) [0x7679f4] 10: (MonitorDBStore::open(std::ostream&)+0xfd) [0x462bfd] 11: main() 12: __libc_start_main() 13: _start() 2026-02-25T22:26:07.188 INFO:scrape:1 jobs: ['70326'] 2026-02-25T22:26:07.188 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu/{latest', 'fs/verify/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mon-debug', 'mount/fuse', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{ignorelist_health', 'overrides}}', 'pg_health', 'ranks/1', 'session_timeout}', 'tasks/dbench', 'validater/valgrind}'] 2026-02-25T22:26:07.188 INFO:scrape: 2026-02-25T22:26:07.188 INFO:scrape:Failure: Test failure: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks) 2026-02-25T22:26:07.188 INFO:scrape:2 jobs: ['70396', '70233'] 2026-02-25T22:26:07.188 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_20.04}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'no_client_pidfile', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/scrub}'] 2026-02-25T22:26:07.188 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_20.04}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/scrub}', 'ubuntu_22.04}'] 2026-02-25T22:26:07.188 INFO:scrape: 2026-02-25T22:26:07.188 INFO:scrape:Failure: Test failure: test_reading_conf (tasks.cephfs.test_cephfs_shell.TestShellOpts.test_reading_conf) 2026-02-25T22:26:07.188 INFO:scrape:1 jobs: ['70378'] 2026-02-25T22:26:07.188 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/ubuntu_20.04', 'fs/shell/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/cephfs-shell}'] 2026-02-25T22:26:07.188 INFO:scrape: 2026-02-25T22:26:07.188 INFO:scrape:Failure: Test failure: test_subvolume_replicated (tasks.cephfs.test_subvolume.TestSubvolumeReplicated.test_subvolume_replicated), test_subvolume_replicated (tasks.cephfs.test_subvolume.TestSubvolumeReplicated.test_subvolume_replicated) 2026-02-25T22:26:07.188 INFO:scrape:1 jobs: ['70398'] 2026-02-25T22:26:07.188 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_latest}', 'fs/volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/volumes/{overrides', 'test/basic}}'] 2026-02-25T22:26:07.188 INFO:scrape: 2026-02-25T22:26:07.188 INFO:scrape:Failure: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm) 2026-02-25T22:26:07.188 INFO:scrape:1 jobs: ['70383'] 2026-02-25T22:26:07.188 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/pool-perm}'] 2026-02-25T22:26:07.188 INFO:scrape: 2026-02-25T22:26:07.188 INFO:scrape:Failure: ["Wanted 2 disks of ~1700 GB (rotational=False), but only matched 1: ['nvme1n1']"] 2026-02-25T22:26:07.188 INFO:scrape:1 jobs: ['70400'] 2026-02-25T22:26:07.188 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-modules}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/sessionmap}', 'ubuntu_22.04}'] 2026-02-25T22:26:07.188 INFO:scrape: 2026-02-25T22:26:07.188 INFO:scrape:Failure: "2026-02-25T16:24:08.437650+0000 mon.a (mon.0) 1996 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log 2026-02-25T22:26:07.188 INFO:scrape:1 jobs: ['70315'] 2026-02-25T22:26:07.188 INFO:scrape:suites: ['conf/{client', 'fs/nfs/{cluster/{1-node}', 'ignorelist_health', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignore_mgr_down', 'pg_health}', 'supported-random-distros$/{ubuntu_20.04}', 'tasks/nfs}'] 2026-02-25T22:26:07.189 INFO:scrape: 2026-02-25T22:26:07.189 INFO:scrape:Failure: Command failed (workunit test suites/dbench.sh) on trial055 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0ef62ce8e48de042e1e5e6a70cfa2ee3f1174d28 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2026-02-25T22:26:07.189 INFO:scrape:1 jobs: ['70391'] 2026-02-25T22:26:07.189 INFO:scrape:suites: ['1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/yes', '3-modules}', '3-snaps/yes', '4-flush/yes', '5-workunit/suites/dbench}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-centos_9.stream', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'n/5', 'objectstore-ec/bluestore-bitmap', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'pg_health', 'ranks/multi/{balancer/automatic', 'replication/default}', 'session_timeout}', 'standby-replay', 'tasks/{0-subvolume/{with-no-extra-options}', 'wsync/yes}'] 2026-02-25T22:26:07.189 INFO:scrape: 2026-02-25T22:26:07.192 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial ================================================================= info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/ logs: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/ failed: 51 dead: 2 running: 0 waiting: 0 queued: 0 passed: 140 Fail ================================================================= [70233] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/scrub} ----------------------------------------------------------------- time: 00:07:39 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70233/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70233/ Test failure: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks) [70241] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:03:01 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70241/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70241/ Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=paci fic [70242] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_9.stream} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/3 tasks/dbench validater/valgrind} ----------------------------------------------------------------- time: 00:44:22 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70242/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70242/ saw valgrind issues [70246] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:31:15 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70246/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70246/ error during thrashing [70247] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-modules} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-client 1-tests/fscrypt-common}} ----------------------------------------------------------------- time: 00:12:07 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70247/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70247/ Test failure: test_fscrypt_dummy_encryption_with_quick_group (tasks.cephfs. test_fscrypt.TestFSCryptXFS.test_fscrypt_dummy_encryption_with_quick_group) [70253] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/async}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70253/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70253/ failed to install new kernel version within timeout [70254] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:21:44 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70254/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70254/ "grep: /var/log/ceph/64d49534-1264-11f1-9de3-d404e6e7d460/ceph.log: No such file or directory" in cluster log [70256] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:09:28 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70256/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70256/ error during thrashing [70269] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 01:29:24 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70269/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70269/ Test failure: test_subvolume_replicated (tasks.cephfs.test_subvolume.TestSu bvolumeReplicated.test_subvolume_replicated) [70270] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:19:12 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70270/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70270/ reached maximum tries (50) after waiting for 300 seconds [70277] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:07:27 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70277/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70277/ error during thrashing [70279] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu/{latest overrides}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/5 tasks/fsstress validater/lockdep} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70279/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70279/ failed to install new kernel version within timeout [70284] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:03:21 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70284/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70284/ Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3cb e25cde3cfa028984618ad32de9edc4c1eaed0 [70285] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:11:03 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70285/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70285/ error during thrashing [70286] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/admin} ----------------------------------------------------------------- time: 00:23:24 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70286/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70286/ Test failure: test_with_health_warn_trim (tasks.cephfs.test_admin.TestFSFail.test_with_health_warn_trim) [70292] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:27:05 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70292/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70292/ "2026-02-25T16:15:11.146345+0000 mds.d (mds.0) 1 : cluster [WRN] client.4628 does not advance its oldest_flush_tid (3923), 100000 completed flushes recorded in session" in cluster log [70294] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/asok_dump_tree} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70294/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70294/ failed to install new kernel version within timeout [70298] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70298/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70298/ failed to install new kernel version within timeout [70302] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:02:53 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70302/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70302/ Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=paci fic [70307] fs/full/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-node-4-mds-1-osd conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/mgr-osd-full} ----------------------------------------------------------------- time: 00:15:53 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70307/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70307/ "2026-02-25T16:21:52.426450+0000 mds.b (mds.2) 1 : cluster [WRN] evicting unresponsive client trial116:x (4632), after 301.982 seconds" in cluster log [70310] fs/mirror/{begin/{0-install 1-ceph 2-logrotate 3-modules} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/mirror} ----------------------------------------------------------------- time: 00:35:09 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70310/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70310/ Test failure: test_cephfs_mirror_cancel_sync (tasks.cephfs.test_mirroring.TestMirroring.test_cephfs_mirror_cancel_sync) [70313] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:47:37 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70313/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70313/ Test failure: test_client_evict_with_invalid_id (tasks.cephfs.test_misc.Tes tSessionClientEvict.test_client_evict_with_invalid_id) [70315] fs/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} overrides/{ignore_mgr_down ignorelist_health pg_health} supported-random-distros$/{ubuntu_20.04} tasks/nfs} ----------------------------------------------------------------- time: 00:27:22 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70315/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70315/ "2026-02-25T16:24:08.437650+0000 mon.a (mon.0) 1996 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log [70321] fs/valgrind/{begin/{0-install 1-ceph 2-logrotate 3-modules} centos_latest debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/{ignorelist_health pg_health} tasks/mirror}} ----------------------------------------------------------------- time: 01:06:35 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70321/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70321/ Test failure: test_cephfs_mirror_incremental_sync_with_type_mixup (tasks.ce phfs.test_mirroring.TestMirroring.test_cephfs_mirror_incremental_sync_with_ type_mixup) [70322] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/finisher_per_module}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70322/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70322/ failed to install new kernel version within timeout [70323] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 01:02:03 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70323/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70323/ error during thrashing [70326] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu/{latest overrides}} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/1 tasks/dbench validater/valgrind} ----------------------------------------------------------------- time: 00:18:06 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70326/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70326/ "grep: /var/log/ceph/ceph.log: No such file or directory" in cluster log [70329] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-recovery} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70329/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70329/ failed to install new kernel version within timeout [70335] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:09:40 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70335/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70335/ error during thrashing [70336] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} ----------------------------------------------------------------- time: 00:09:29 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70336/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70336/ Test failure: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction) [70339] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:04:04 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70339/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70339/ "grep: /var/log/ceph/3c0a34fe-1266-11f1-9f22-d404e6e7d460/ceph.log: No such file or directory" in cluster log [70342] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:08:40 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70342/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70342/ error during thrashing [70350] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} ----------------------------------------------------------------- time: 00:31:30 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70350/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70350/ Command failed (workunit test fs/misc/fallocate.sh) on trial026 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0ef62ce8e48de042e1e5e6a70cfa2ee3f1174d28 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/fallocate.sh' [70362] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70362/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70362/ failed to install new kernel version within timeout [70367] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:25:39 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70367/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70367/ error during thrashing [70369] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} ----------------------------------------------------------------- time: 00:30:58 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70369/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70369/ "2026-02-25T16:45:17.767746+0000 mds.b (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/36804114-ab2d-47f0-8d93- 798a4d140c38/postgres/data/pg_wal) see mds.b log and `damage ls` output for details" in cluster log [70371] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:08:53 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70371/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70371/ error during thrashing [70374] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:02:50 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70374/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70374/ Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=paci fic [70378] fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_20.04 mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} ----------------------------------------------------------------- time: 00:29:19 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70378/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70378/ Test failure: test_reading_conf (tasks.cephfs.test_cephfs_shell.TestShellOpts.test_reading_conf) [70382] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:25:21 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70382/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70382/ error during thrashing [70383] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/pool-perm} ----------------------------------------------------------------- time: 00:07:22 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70383/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70383/ Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm) [70390] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70390/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70390/ failed to install new kernel version within timeout [70391] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} ----------------------------------------------------------------- time: 00:38:31 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70391/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70391/ Command failed (workunit test suites/dbench.sh) on trial055 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0ef62ce8e48de042e1e5e6a70cfa2ee3f1174d28 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' [70393] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:04:12 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70393/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70393/ "grep: /var/log/ceph/66f6588d-126a-11f1-bb93-d404e6e7d460/ceph.log: No such file or directory" in cluster log [70395] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_9.stream} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/5 tasks/dbench validater/valgrind} ----------------------------------------------------------------- time: 00:37:41 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70395/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70395/ saw valgrind issues [70396] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/scrub} ----------------------------------------------------------------- time: 00:07:33 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70396/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70396/ Test failure: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks) [70398] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 01:47:37 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70398/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70398/ Test failure: test_subvolume_replicated (tasks.cephfs.test_subvolume.TestSu bvolumeReplicated.test_subvolume_replicated), test_subvolume_replicated (ta sks.cephfs.test_subvolume.TestSubvolumeReplicated.test_subvolume_replicated ) [70399] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:10:07 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70399/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70399/ error during thrashing [70403] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snap_schedule_snapdir} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70403/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70403/ failed to install new kernel version within timeout [70410] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:47:56 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70410/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70410/ Test failure: test_client_evict_with_invalid_id (tasks.cephfs.test_misc.Tes tSessionClientEvict.test_client_evict_with_invalid_id) [70418] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 00:08:45 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70418/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70418/ error during thrashing Dead ================================================================= [70360] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/kernel_untar_build}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70360/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70360/ killed [70400] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/sessionmap} ----------------------------------------------------------------- time: 00:02:34 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70400/ log: https://qa-proxy.ceph.com/teuthology/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70400/ ["Wanted 2 disks of ~1700 GB (rotational=False), but only matched 1: ['nvme1n1']"] Pass ================================================================= [70232] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/ffsb}} time: 00:21:42 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70232/ [70234] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/iozone}} time: 00:09:02 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70234/ [70235] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/snaps}} time: 00:34:30 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70235/ [70236] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/sessionmap} time: 00:09:38 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70236/ [70237] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress} time: 00:09:04 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70237/ [70238] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_9.stream} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:20:35 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70238/ [70239] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_misc} time: 00:08:21 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70239/ [70240] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:33:04 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70240/ [70243] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} time: 01:10:04 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70243/ [70244] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snap_schedule_snapdir} time: 00:06:59 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70244/ [70245] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/norstats}} time: 00:24:21 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70245/ [70248] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snapshots} time: 00:23:39 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70248/ [70249] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/ior-shared-file} time: 00:07:04 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70249/ [70250] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/multifs-auth} time: 00:14:24 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70250/ [70251] fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} time: 00:29:27 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70251/ [70252] fs/snaps/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} time: 00:17:43 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70252/ [70255] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/strays} time: 00:26:09 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70255/ [70257] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:21:58 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70257/ [70258] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsstress}} time: 00:13:51 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70258/ [70259] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/test_journal_migration} time: 00:09:36 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70259/ [70260] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/fsstress}} time: 00:10:03 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70260/ [70261] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/test_snap_schedule/{overrides/{v1} snap-schedule}} time: 02:08:49 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70261/ [70262] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:07:48 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70262/ [70263] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/libcephfs_python} time: 00:08:40 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70263/ [70264] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:07:59 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70264/ [70265] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-clients/kclient 3-workload/stress_tests/blogbench}} time: 00:21:39 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70265/ [70266] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:16:12 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70266/ [70267] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/truncate_delay} time: 00:09:11 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70267/ [70268] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} time: 00:27:23 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70268/ [70271] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/workunit/dir-max-entries} time: 00:08:39 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70271/ [70272] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} time: 00:06:34 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70272/ [70273] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/dbench}} time: 00:51:11 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70273/ [70274] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/workunit/quota} time: 00:06:39 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70274/ [70275] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{latest overrides}} kclient-overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:42:08 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70275/ [70276] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:08:11 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70276/ [70278] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsync-tester}} time: 00:12:16 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70278/ [70280] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/acls} time: 00:09:06 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70280/ [70281] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/mdtest} time: 00:08:33 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70281/ [70282] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/failover} time: 00:41:20 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70282/ [70283] fs/snaps/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} time: 00:26:59 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70283/ [70287] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-compat_client/yes}} time: 00:11:42 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70287/ [70288] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress} time: 00:09:10 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70288/ [70289] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_misc} time: 00:07:44 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70289/ [70290] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/test_o_trunc}} time: 00:19:54 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70290/ [70291] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/alternate-pool} time: 00:06:06 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70291/ [70293] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone}} time: 00:41:43 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70293/ [70295] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:08:48 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70295/ [70296] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/auto-repair} time: 00:07:29 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70296/ [70297] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}} time: 00:23:09 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70297/ [70299] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/backtrace} time: 00:06:38 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70299/ [70300] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cap-flush} time: 00:08:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70300/ [70301] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} time: 00:26:52 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70301/ [70303] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/iozone}} time: 00:26:15 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70303/ [70304] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress} time: 00:08:30 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70304/ [70305] fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate 3-modules} centos_latest clusters/small-cluster conf/{client mds mgr mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/trim-i24137} time: 00:06:55 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70305/ [70306] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench conf/{client mds mgr mon osd} distro/single-container-host overrides/{ignorelist_health pg_health}} time: 01:03:56 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70306/ [70308] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-limits} time: 00:18:09 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70308/ [70309] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client} time: 00:08:50 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70309/ [70311] fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate 3-modules} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} conf/{client mds mgr mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distro$/{ubuntu_latest} workloads/cephfs-mirror-ha-workunit} time: 00:22:11 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70311/ [70312] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_9.stream} kclient-overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:35:42 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70312/ [70314] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/multifs-auth} time: 00:24:39 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70314/ [70316] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_misc} time: 00:07:30 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70316/ [70317] fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/centos_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} time: 00:30:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70317/ [70318] fs/snaps/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} time: 00:24:25 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70318/ [70319] fs/top/{begin/{0-install 1-ceph 2-logrotate 3-modules} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/fstop} time: 00:06:54 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70319/ [70320] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_blogbench traceless/50pc} time: 00:17:41 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70320/ [70324] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-compat_client/no}} time: 00:13:25 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70324/ [70325] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-readahead} time: 00:06:02 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70325/ [70327] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} time: 00:16:57 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70327/ [70328] fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}} time: 00:05:32 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70328/ [70330] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/pjd}} time: 00:13:36 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70330/ [70331] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-modules} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-client 1-tests/fscrypt-iozone}} time: 00:13:57 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70331/ [70332] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/damage} time: 00:16:19 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70332/ [70333] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} time: 00:09:44 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70333/ [70334] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/data-scan} time: 00:29:11 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70334/ [70337] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/direct_io}} time: 00:11:40 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70337/ [70338] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/exports} time: 00:44:47 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70338/ [70340] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:07:59 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70340/ [70341] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:08:19 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70341/ [70343] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/forward-scrub} time: 00:14:08 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70343/ [70344] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-clients/kclient 3-workload/stress_tests/dbench}} time: 00:43:13 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70344/ [70345] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/trivial_sync}} time: 00:06:36 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70345/ [70346] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/fragment} time: 00:11:55 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70346/ [70347] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/ior-shared-file} time: 00:06:41 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70347/ [70348] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/failover} time: 00:37:33 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70348/ [70349] fs/snaps/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} time: 00:23:14 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70349/ [70351] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:16:26 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70351/ [70352] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{latest overrides}} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:24:05 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70352/ [70353] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:48:47 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70353/ [70354] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/journal-repair} time: 00:17:17 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70354/ [70355] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} time: 00:09:28 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70355/ [70356] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:13:22 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70356/ [70357] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/mds-flush} time: 00:06:16 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70357/ [70358] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/snapshot}} time: 00:22:00 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70358/ [70359] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_9.stream} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/3 tasks/fsstress validater/lockdep} time: 00:10:30 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70359/ [70361] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/fsstress}} time: 00:22:36 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70361/ [70363] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/mds-full} time: 00:14:45 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70363/ [70364] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress} time: 00:08:04 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70364/ [70365] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/ino_release_cb} time: 00:06:54 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70365/ [70366] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_misc} time: 00:07:42 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70366/ [70368] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/mds_creation_retry} time: 00:10:19 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70368/ [70370] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/metrics} time: 00:18:01 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70370/ [70372] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/multimds_misc} time: 00:11:35 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70372/ [70373] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} time: 00:09:27 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70373/ [70375] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/openfiletable} time: 00:09:51 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70375/ [70376] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/mdtest} time: 00:07:54 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70376/ [70377] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/multifs-auth} time: 00:25:20 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70377/ [70379] fs/snaps/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} time: 00:26:48 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70379/ [70380] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/async}} time: 00:09:52 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70380/ [70381] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} time: 00:20:21 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70381/ [70384] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-compat_client/yes}} time: 00:12:11 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70384/ [70385] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:07:41 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70385/ [70386] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{latest overrides}} kclient-overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} time: 00:39:40 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70386/ [70387] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:08:10 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70387/ [70388] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:26:50 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70388/ [70389] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/quota} time: 00:07:31 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70389/ [70392] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/recovery-fs} time: 00:07:30 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70392/ [70394] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} time: 00:16:28 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70394/ [70397] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/quincy 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/blogbench}} time: 00:17:48 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70397/ [70401] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/ffsb}} time: 00:30:31 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70401/ [70402] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-modules} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-client 1-tests/fscrypt-dbench}} time: 03:43:06 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70402/ [70404] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:10:07 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70404/ [70405] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:13:45 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70405/ [70406] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:07:57 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70406/ [70407] fs/cephadm/renamevolume/{0-start 1-rename conf/{client mds mgr mon osd} distro/single-container-host overrides/{ignorelist_health pg_health}} time: 00:07:46 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70407/ [70408] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snapshots} time: 00:24:10 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70408/ [70409] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/libcephfs/{frag test}} time: 01:11:45 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70409/ [70411] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/failover} time: 00:38:36 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70411/ [70412] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:07:25 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70412/ [70413] fs/snaps/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} time: 00:34:57 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70413/ [70414] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:09:28 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70414/ [70415] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/quincy 1-client 2-upgrade 3-compat_client/no}} time: 00:16:37 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70415/ [70416] fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/norstats}} time: 00:21:55 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70416/ [70417] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock ubuntu_22.04} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/strays} time: 00:19:01 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70417/ [70419] fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/quincy 1-upgrade}} time: 00:12:35 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70419/ [70420] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/test_journal_migration} time: 00:12:25 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70420/ [70421] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone}} time: 00:42:49 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70421/ [70422] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{latest overrides}} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_untarbuild_blogbench} time: 00:23:02 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70422/ [70423] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:09:20 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70423/ [70424] fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/test_snap_schedule/{overrides/{v2} snap-schedule}} time: 02:11:30 info: https://pulpito.ceph.com/yuriw-2026-02-25_15:27:50-fs-reef-release-distro-default-trial/70424/ 2026-02-25T22:26:07.200 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 33, in main results(args['--archive-dir'], args['--name'], args['--email'], File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 78, in results email_results( File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/results.py", line 95, in email_results smtp = smtplib.SMTP('localhost') ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/smtplib.py", line 255, in __init__ (code, msg) = self.connect(host, port) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/smtplib.py", line 341, in connect self.sock = self._get_socket(host, port, self.timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/smtplib.py", line 312, in _get_socket return socket.create_connection((host, port), timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/socket.py", line 115, in create_connection sock.connect(sa) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 590, in connect self._internal_connect(address) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 634, in _internal_connect raise _SocketError(err, strerror(err)) ConnectionRefusedError: [Errno 111] Connection refused