2026-02-13T01:54:42.399 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-13T01:54:42.399 INFO:teuthology.results:subset: '604/1024' 2026-02-13T01:54:42.399 INFO:teuthology.results:seed: '7166' 2026-02-13T04:55:55.844 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-13T04:55:55.851 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2026-02-13T16:56:46.081 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2026-02-13T16:56:52.093 INFO:scrape:Found 241 jobs 2026-02-13T16:56:55.468 WARNING:scrape:Missing teuthology log /home/teuthworker/mnt/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48017/teuthology.log 2026-02-13T16:58:11.583 INFO:scrape:Found 32 distinct failure reasons 2026-02-13T16:58:11.583 INFO:scrape:Failure: error during thrashing 2026-02-13T16:58:11.583 INFO:scrape:2 jobs: ['48103', '48068'] 2026-02-13T16:58:11.583 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'conf/{client', 'distro/{rpm_latest}', 'frag', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{client-shutdown', 'pg_health', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}'] 2026-02-13T16:58:11.583 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '2-workunit/cfuse_workunit_suites_pjd}}', '2-workunit/suites/pjd}}', '3-kernel}', 'clusters/1a3s-mds-2c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rpm_latest}', 'frag', 'fs/thrash/multifs/{begin/{0-install', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{client-shutdown', 'overrides/{distro/stock/{k-stock', 'pg_health', 'prefetch_entire_dirfrags/no', 'prefetch_oft_dirfrags/no', 'races', 'ranks/5', 'rpm_latest}', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health}'] 2026-02-13T16:58:11.583 INFO:scrape: 2026-02-13T16:58:11.583 INFO:scrape:Failure: "2026-02-13T02:22:45.753103+0000 mon.trial127 (mon.0) 375 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log 2026-02-13T16:58:11.583 INFO:scrape:62 jobs 2026-02-13T16:58:11.583 INFO:scrape:suites intersection: ['conf/{client', 'mds', 'mgr', 'mon', 'osd}'] 2026-02-13T16:58:11.583 INFO:scrape:suites union: ['0-subvolume/{with-namespace-isolated-and-quota}', '0-subvolume/{with-namespace-isolated}', '0-subvolume/{with-no-extra-options}', '0-subvolume/{with-quota}', '0-use-snaprealm/{global}', '0-use-snaprealm/{subvol}', '1-cephadm', '1-check-counter', '1-mount', '1-rename', '2-logrotate', '2-scrub/no', '2-scrub/yes', '2-workload/dbench', '3-kernel}', '3-snaps/no', '3-snaps/yes', '4-flush/no', '4-flush/yes', '5-quiesce/no', '5-quiesce/with-quiesce', '6-workunit/direct_io}}', '6-workunit/fs/misc}}', '6-workunit/fs/norstats}}', '6-workunit/fs/test_o_trunc}}', '6-workunit/kernel_untar_build}}', '6-workunit/postgres}}', '6-workunit/suites/blogbench}}', '6-workunit/suites/dbench}}', '6-workunit/suites/ffsb}}', '6-workunit/suites/fsstress}}', '6-workunit/suites/fsx}}', '6-workunit/suites/fsync-tester}}', '6-workunit/suites/iogen}}', '6-workunit/suites/iozone}}', '6-workunit/suites/pjd}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'distro/single-container-host', 'export-check', 'frag', 'fs/cephadm/multivolume/{0-start', 'fs/cephadm/renamevolume/{0-start', 'fs/workload/{0-distro', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{base/{mount-syntax/{v1}', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/crc', 'ms_mode/legacy', 'ms_mode/secure', 'n/3', 'n/5', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg_health', 'pg_health}}', 'ranks/1', 'ranks/multi/{balancer/automatic', 'ranks/multi/{balancer/random', 'replication/always}', 'replication/default}', 'rpm_latest}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}', 'wsync/no}', 'wsync/yes}'] 2026-02-13T16:58:11.583 INFO:scrape: 2026-02-13T16:58:11.584 INFO:scrape:Failure: Command failed (workunit test fs/snaps/untar_snap_rm.sh) on trial025 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' 2026-02-13T16:58:11.584 INFO:scrape:5 jobs: ['48153', '48163', '48033', '48077', '47993'] 2026-02-13T16:58:11.584 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}'] 2026-02-13T16:58:11.584 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '2-workunit/cfuse_workunit_snaptests}}', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-1c-client', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{rpm_latest}', 'distro/{ubuntu_latest}', 'frag', 'fs/snaps/{0-use-snaprealm/{global}', 'fs/snaps/{0-use-snaprealm/{subvol}', 'fs/thrash/multifs/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'multifs', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{client-shutdown', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'pg_health', 'pg_health}', 'rpm_latest}', 'session_timeout', 'tasks/workunit/snaps}', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'thrashosds-health}'] 2026-02-13T16:58:11.584 INFO:scrape: 2026-02-13T16:58:11.584 INFO:scrape:Failure: Command failed (workunit test fs/misc/chmod.sh) on trial082 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh' 2026-02-13T16:58:11.584 INFO:scrape:2 jobs: ['47991', '48054'] 2026-02-13T16:58:11.584 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/permission/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/cfuse_workunit_misc}'] 2026-02-13T16:58:11.584 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/permission/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/cfuse_workunit_misc}'] 2026-02-13T16:58:11.584 INFO:scrape: 2026-02-13T16:58:11.584 INFO:scrape:Failure: Command failed (workunit test client/test_oc_disabled.sh) on trial091 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/client/test_oc_disabled.sh' 2026-02-13T16:58:11.584 INFO:scrape:1 jobs: ['47984'] 2026-02-13T16:58:11.584 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/libcephfs/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/client}'] 2026-02-13T16:58:11.584 INFO:scrape: 2026-02-13T16:58:11.584 INFO:scrape:Failure: Command failed on trial168 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback' 2026-02-13T16:58:11.584 INFO:scrape:6 jobs: ['47952', '48008', '48025', '48085', '48133', '48148'] 2026-02-13T16:58:11.584 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'osd}'] 2026-02-13T16:58:11.584 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '2-workunit/cfuse_workunit_trivial_sync}}', '2-workunit/ffsb}}', '3-kernel}', 'begin/{0-install', 'bluestore-bitmap', 'clusters/1-mds-1-client', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu/{overrides', 'distro/{ubuntu_latest}', 'frag', 'fs/fscrypt/{begin/{0-install', 'fs/functional/{0-use-snaprealm/{subvol}', 'fs/thrash/multifs/{begin/{0-install', 'fs/verify/{begin/{0-install', 'ignorelist_health', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mon-debug', 'mount', 'mount/kclient/{k-testing', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'multifs', 'no_client_pidfile', 'objectstore-ec/bluestore-bitmap', 'objectstore/bluestore-bitmap', 'osd', 'osd}', 'overrides/{client-shutdown', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'pg_health}', 'ranks/3', 'ranks/5', 'session_timeout', 'session_timeout}', 'tasks/fsstress', 'tasks/snapshots}', 'tasks/{1-tests/fscrypt-iozone}}', 'tasks/{1-thrash/mds', 'thrashosds-health}', 'ubuntu_latest}}', 'validater/lockdep}'] 2026-02-13T16:58:11.584 INFO:scrape: 2026-02-13T16:58:11.584 INFO:scrape:Dead: None 2026-02-13T16:58:11.584 INFO:scrape:4 jobs: ['48017', '48105', '48016', '48010'] 2026-02-13T16:58:11.584 INFO:scrape: 2026-02-13T16:58:11.584 INFO:scrape:Dead: b'2026-02-13T09:06:01.680 DEBUG:teuthology.exit:Finished running handlers' 2026-02-13T16:58:11.585 INFO:scrape:15 jobs: ['48066', '47996', '48078', '48123', '48070', '47948', '47972', '48130', '48116', '48113', '47987', '48051', '48058', '48071', '48004'] 2026-02-13T16:58:11.585 INFO:scrape:suites intersection: ['conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}'] 2026-02-13T16:58:11.585 INFO:scrape:suites union: ['1-ceph', '1-mount/mount/kclient/{mount-syntax/{v1}', '2-clients/kclient', '2-logrotate', '2-workunit/ffsb}}', '2-workunit/fs/snaps}}', '3-kernel}', '3-workload/new_ops/{newops}}}', 'begin/{0-install', 'bluestore-bitmap', 'centos_9.stream', 'clusters/1-mds-1-client', 'clusters/1a3s-mds-1c-client', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'clusters/1a5s-mds-1c-client', 'clusters/{1-mds-1-client-micro}', 'conf/{client', 'distro/{rpm_latest}', 'distro/{ubuntu/{overrides', 'distro/{ubuntu_latest}', 'frag', 'fs/fscrypt/{begin/{0-install', 'fs/functional/{0-use-snaprealm/{global}', 'fs/functional/{0-use-snaprealm/{subvol}', 'fs/mixed-clients/{begin/{0-install', 'fs/multifs/{begin/{0-install', 'fs/snaps/{0-use-snaprealm/{subvol}', 'fs/thrash/multifs/{begin/{0-install', 'fs/thrash/workloads/{begin/{0-install', 'fs/upgrade/upgraded_client/{bluestore-bitmap', 'fs/volumes/{begin/{0-install', 'ignorelist_health', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/testing/k-testing', 'kernel', 'mds', 'mgr', 'mon', 'mon-debug', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'multifs', 'no_client_pidfile', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd', 'osd-asserts', 'osd}', 'overrides', 'overrides/{client-shutdown', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'pg_health}', 'prefetch_entire_dirfrags/no', 'prefetch_oft_dirfrags/yes', 'races', 'ranks/1', 'rpm_latest}', 'session_timeout', 'tasks/acls}', 'tasks/client-recovery}', 'tasks/failover}', 'tasks/fscrypt-kbuild}', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'tasks/quiesce}', 'tasks/quota}', 'tasks/referent}', 'tasks/volumes/{0-use-snaprealm/{global}', 'tasks/workunit/snaps}', 'tasks/xfstests-dev}', 'tasks/{0-install/squid', 'tasks/{1-tests/fscrypt-common}}', 'tasks/{1-thrash/mds', 'test/async}}', 'thrashosds-health}', 'ubuntu_latest}}'] 2026-02-13T16:58:11.585 INFO:scrape: 2026-02-13T16:58:11.585 INFO:scrape:Failure: Fuse mount failed to populate/sys/ after 31 seconds 2026-02-13T16:58:11.585 INFO:scrape:17 jobs: ['48011', '47989', '48072', '48187', '48112', '47974', '47988', '48001', '48100', '48031', '48151', '48143', '48150', '48091', '48185', '47998', '48044'] 2026-02-13T16:58:11.585 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}'] 2026-02-13T16:58:11.585 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '2-workunit/cfuse_workunit_suites_fsstress}}', '3-kernel}', 'begin/{0-install', 'clusters/1-mds-3-client', 'clusters/1a3s-mds-2c-client', 'clusters/1a3s-mds-4c-client', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{ubuntu_latest}', 'distros/ubuntu_latest', 'frag', 'fs/functional/{0-use-snaprealm/{global}', 'fs/functional/{0-use-snaprealm/{subvol}', 'fs/multiclient/{begin/{0-install', 'fs/multifs/{begin/{0-install', 'fs/permission/{begin/{0-install', 'fs/thrash/multifs/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mon-debug', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'no_client_pidfile', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{client-shutdown', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'pg_health', 'pg_health}', 'session_timeout', 'snap-schedule}}', 'tasks/auto-repair}', 'tasks/cephfs_misc_tests}', 'tasks/cfuse_workunit_misc}', 'tasks/damage}', 'tasks/data-scan}', 'tasks/forward-scrub}', 'tasks/ior-shared-file}', 'tasks/journal-repair}', 'tasks/mdtest}', 'tasks/multifs-auth}', 'tasks/openfiletable}', 'tasks/strays}', 'tasks/test_snap_schedule/{overrides/{v2}', 'tasks/{1-thrash/mds', 'thrashosds-health}'] 2026-02-13T16:58:11.585 INFO:scrape: 2026-02-13T16:58:11.585 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=rocky%2F10%2Fx86_64&sha1=0eceb0defba60152a8182f7bd87d164b639885b8 2026-02-13T16:58:11.585 INFO:scrape:2 jobs: ['48040', '48135'] 2026-02-13T16:58:11.585 INFO:scrape:suites intersection: ['2-test_admin_squid_client}}', 'conf/{client', 'fs/bugs/multifs_mdsauthcaps/{clusters/1node-cluster-1node-client', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/{0-install'] 2026-02-13T16:58:11.585 INFO:scrape:suites union: ['1-mount/fuse', '1-mount/kclient/{mount-syntax/{v1}', '2-test_admin_squid_client}}', 'conf/{client', 'distro/{rpm_latest}', 'distro/{ubuntu_latest}', 'fs/bugs/multifs_mdsauthcaps/{clusters/1node-cluster-1node-client', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'pg_health}', 'rpm_latest}', 'tasks/{0-install'] 2026-02-13T16:58:11.585 INFO:scrape: 2026-02-13T16:58:11.585 INFO:scrape:Failure: Test failure: test_clone_after_subvol_is_removed (tasks.cephfs.volumes.test_clone_stats.TestCloneProgressReporter.test_clone_after_subvol_is_removed) 2026-02-13T16:58:11.585 INFO:scrape:1 jobs: ['48047'] 2026-02-13T16:58:11.585 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rpm_latest}', 'fs/volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/volumes/{0-use-snaprealm/{global}', 'test/clone-progress}}'] 2026-02-13T16:58:11.585 INFO:scrape: 2026-02-13T16:58:11.585 INFO:scrape:Failure: Command failed (workunit test suites/blogbench.sh) on trial015 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh' 2026-02-13T16:58:11.585 INFO:scrape:4 jobs: ['48069', '48186', '48035', '48139'] 2026-02-13T16:58:11.585 INFO:scrape:suites intersection: ['2-logrotate', '3-kernel}', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}'] 2026-02-13T16:58:11.585 INFO:scrape:suites union: ['0-subvolume/{with-namespace-isolated}', '0-subvolume/{with-no-extra-options}', '0-use-snaprealm/{global}', '0-use-snaprealm/{subvol}', '1-ceph', '1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/yes', '3-kernel}', '3-snaps/no', '3-snaps/yes', '4-flush/no', '4-flush/yes', '5-quiesce/no', '6-workunit/suites/blogbench}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'clusters/1a3s-mds-2c-client', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{rpm_latest}', 'distro/{ubuntu/{overrides', 'export-check', 'frag', 'fs/mixed-clients/{begin/{0-install', 'fs/traceless/{begin/{0-install', 'fs/workload/{0-distro', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/stock/{k-stock', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{base/{mount-syntax/{v1}', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'n/3', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'overrides/{frag', 'overrides/{ignorelist_health', 'pg_health', 'pg_health}', 'ranks/multi/{balancer/automatic', 'ranks/multi/{balancer/random', 'replication/always}', 'replication/default}', 'rpm_latest}', 'session_timeout}', 'standby-replay', 'tasks/cfuse_workunit_suites_blogbench', 'tasks/kernel_cfuse_workunits_untarbuild_blogbench}', 'tasks/{0-fscrypt/{with-fscrypt-dummy}', 'traceless/50pc}', 'ubuntu_latest}}', 'wsync/yes}'] 2026-02-13T16:58:11.585 INFO:scrape: 2026-02-13T16:58:11.586 INFO:scrape:Failure: Command failed (workunit test suites/iozone.sh) on trial071 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/iozone.sh' 2026-02-13T16:58:11.586 INFO:scrape:7 jobs: ['48056', '48063', '48030', '47965', '48141', '47960', '48147'] 2026-02-13T16:58:11.586 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'conf/{client', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd}'] 2026-02-13T16:58:11.586 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '2-workunit/iozone}}', '2-workunit/suites/iozone}}', '3-kernel}', 'clusters/1a3s-mds-2c-client', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rpm_latest}', 'distro/{ubuntu_latest}', 'frag', 'fs/mixed-clients/{begin/{0-install', 'fs/thrash/multifs/{begin/{0-install', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/stock/{k-stock', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-ec-root', 'objectstore/bluestore-bitmap', 'osd-asserts', 'osd}', 'overrides/{client-shutdown', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'pg_health', 'pg_health}', 'prefetch_entire_dirfrags/no', 'prefetch_entire_dirfrags/yes', 'prefetch_oft_dirfrags/yes', 'races', 'ranks/3', 'rpm_latest}', 'session_timeout', 'tasks/kernel_cfuse_workunits_dbench_iozone}', 'tasks/{1-thrash/exports', 'tasks/{1-thrash/mds', 'tasks/{1-thrash/mon', 'tasks/{1-thrash/with-quiesce', 'thrashosds-health}'] 2026-02-13T16:58:11.586 INFO:scrape: 2026-02-13T16:58:11.586 INFO:scrape:Failure: Test failure: test_get_doesnt_create_dir (tasks.cephfs.test_cephfs_shell.TestGetAndPut.test_get_doesnt_create_dir) 2026-02-13T16:58:11.586 INFO:scrape:1 jobs: ['48115'] 2026-02-13T16:58:11.586 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/ubuntu_latest', 'fs/shell/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/cephfs-shell}'] 2026-02-13T16:58:11.586 INFO:scrape: 2026-02-13T16:58:11.586 INFO:scrape:Failure: Command failed (workunit test fs/fscrypt.sh) on trial189 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh none iozone' 2026-02-13T16:58:11.586 INFO:scrape:3 jobs: ['48166', '48041', '48136'] 2026-02-13T16:58:11.586 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'bluestore-bitmap', 'clusters/1-mds-1-client', 'conf/{client', 'fs/fscrypt/{begin/{0-install', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'osd', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health}'] 2026-02-13T16:58:11.586 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '3-kernel}', 'bluestore-bitmap', 'clusters/1-mds-1-client', 'conf/{client', 'distro/{rpm_latest}', 'distro/{ubuntu_latest}', 'fs/fscrypt/{begin/{0-install', 'ignorelist_health_more', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'osd', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health}', 'rpm_latest}', 'tasks/{1-tests/fscrypt-ffsb}}', 'tasks/{1-tests/fscrypt-iozone}}', 'tasks/{1-tests/fscrypt-pjd}}'] 2026-02-13T16:58:11.586 INFO:scrape: 2026-02-13T16:58:11.586 INFO:scrape:Failure: 2026-02-13T16:58:11.586 INFO:scrape:4 jobs: ['48037', '48106', '47961', '48180'] 2026-02-13T16:58:11.586 INFO:scrape:suites intersection: ['1-volume/{0-create', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}', 'tasks/{0-from/tentacle'] 2026-02-13T16:58:11.586 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client/fuse', '2-client/kclient', '3-inline/no', '3-inline/yes', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/no', 'fail_fs/yes', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}', 'tasks/{0-from/tentacle'] 2026-02-13T16:58:11.586 INFO:scrape: 2026-02-13T16:58:11.586 INFO:scrape:Failure: Command failed on trial138 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8223b82f-0889-11f1-b4d7-d404e6e7d460 -e sha1=8f8d38294e4db8750b4b1649f62ce37caa63080f -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' 2026-02-13T16:58:11.586 INFO:scrape:3 jobs: ['48064', '48157', '48013'] 2026-02-13T16:58:11.586 INFO:scrape:suites intersection: ['1-volume/{0-create', '2-client/fuse', '3-inline/no', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/yes', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}'] 2026-02-13T16:58:11.586 INFO:scrape:suites union: ['1-ranks/1', '1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-allow_standby_replay/yes', '2-client/fuse', '3-inline/no', '3-upgrade-mgr-staggered', '4-config-upgrade/{fail_fs}', '4-verify}', '5-upgrade-with-workload', '6-verify}}', 'centos_9.stream', 'conf/{client', 'fail_fs/yes', 'fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap', 'ignorelist_upgrade', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health', 'roles', 'syntax}', 'tasks/{0-from/reef/{v18.2.0}', 'tasks/{0-from/squid'] 2026-02-13T16:58:11.586 INFO:scrape: 2026-02-13T16:58:11.586 INFO:scrape:Failure: Command failed on trial169 with status 1: 'sudo yum -y install ceph-osd-classic' 2026-02-13T16:58:11.587 INFO:scrape:4 jobs: ['48060', '48092', '48028', '48174'] 2026-02-13T16:58:11.587 INFO:scrape:suites intersection: ['centos_9.stream', 'conf/{client', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health}'] 2026-02-13T16:58:11.587 INFO:scrape:suites union: ['1-mount/mount/fuse', '1-upgrade}}', '2-clients/fuse-upgrade', '3-workload/stress_tests/blogbench}}', '3-workload/stress_tests/fsstress}}', 'centos_9.stream', 'clusters/{1-mds-1-client-micro}', 'conf/{client', 'fs/upgrade/nofs/{bluestore-bitmap', 'fs/upgrade/upgraded_client/{bluestore-bitmap', 'ignorelist_wrongly_marked_down', 'kernel', 'mds', 'mgr', 'mon', 'no-mds-cluster', 'osd}', 'overrides/{ignorelist_health', 'pg-warn', 'pg_health}', 'tasks/{0-from/tentacle', 'tasks/{0-install/tentacle'] 2026-02-13T16:58:11.587 INFO:scrape: 2026-02-13T16:58:11.587 INFO:scrape:Failure: reached maximum tries (50) after waiting for 300 seconds 2026-02-13T16:58:11.587 INFO:scrape:4 jobs: ['48170', '48020', '48109', '48096'] 2026-02-13T16:58:11.587 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'conf/{client', 'distro/{ubuntu_latest}', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'osd}'] 2026-02-13T16:58:11.587 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '2-workunit/suites/ffsb}}', '3-kernel}', 'clusters/1a5s-mds-1c-client', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{ubuntu_latest}', 'frag', 'fs/32bits/{begin/{0-install', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{client-shutdown', 'overrides/{faked-ino', 'pg_health', 'pg_health}', 'prefetch_entire_dirfrags/yes', 'prefetch_oft_dirfrags/no', 'races', 'ranks/3', 'session_timeout', 'tasks/cfuse_workunit_suites_pjd}', 'tasks/{1-thrash/osd', 'thrashosds-health}'] 2026-02-13T16:58:11.587 INFO:scrape: 2026-02-13T16:58:11.587 INFO:scrape:Failure: Command failed (workunit test fs/fscrypt_kclient_fuse.sh) on trial126 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/fscrypt-fuse && cd -- /home/ubuntu/cephtest/mnt.0/fscrypt-fuse && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 4h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt_kclient_fuse.sh fuse' 2026-02-13T16:58:11.587 INFO:scrape:1 jobs: ['48111'] 2026-02-13T16:58:11.587 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{rpm_latest}', 'fs/mixed-clients/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'kclient-overrides/{distro/stock/{k-stock', 'mds', 'mgr', 'mon', 'ms-die-on-skipped}', 'objectstore-ec/bluestore-comp-ec-root', 'osd-asserts', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'rpm_latest}', 'tasks/fscrypt-kbuild}'] 2026-02-13T16:58:11.587 INFO:scrape: 2026-02-13T16:58:11.587 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/fs/misc/trivial_sync.sh 2026-02-13T16:58:11.587 INFO:scrape:1 jobs: ['48049'] 2026-02-13T16:58:11.587 INFO:scrape:suites: ['1-ceph', '2-logrotate', '2-workunit/fs/trivial_sync}}', '3-kernel}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'frag', 'fs/thrash/workloads/{begin/{0-install', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{client-shutdown', 'overrides/{distro/testing/k-testing', 'pg_health', 'prefetch_entire_dirfrags/yes', 'prefetch_oft_dirfrags/no', 'races', 'ranks/1', 'session_timeout', 'tasks/{1-thrash/with-quiesce', 'thrashosds-health}'] 2026-02-13T16:58:11.587 INFO:scrape: 2026-02-13T16:58:11.587 INFO:scrape:Failure: Test failure: test_add_ancestor_and_child_directory (tasks.cephfs.test_mirroring.TestMirroring.test_add_ancestor_and_child_directory) 2026-02-13T16:58:11.587 INFO:scrape:1 jobs: ['47995'] 2026-02-13T16:58:11.587 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'clients/mirror', 'cluster/1-node', 'debug', 'fs/valgrind/{begin/{0-install', 'mirror/{cephfs-mirror/one-per-cluster', 'mount/fuse', 'overrides/{ignorelist_health', 'pg_health}', 'rpm_latest}', 'tasks/mirror}'] 2026-02-13T16:58:11.587 INFO:scrape: 2026-02-13T16:58:11.587 INFO:scrape:Failure: Command failed on trial077 with status 1: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"\'' 2026-02-13T16:58:11.587 INFO:scrape:1 jobs: ['48138'] 2026-02-13T16:58:11.587 INFO:scrape:suites: ['0-subvolume/{with-namespace-isolated}', '0-use-snaprealm/{global}', '1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/no', '3-kernel}', '3-snaps/no', '4-flush/yes', '5-quiesce/with-quiesce', '6-workunit/postgres}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-distro', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'ms_mode/secure', 'n/5', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'pg_health', 'ranks/multi/{balancer/random', 'replication/always}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}', 'wsync/no}'] 2026-02-13T16:58:11.587 INFO:scrape: 2026-02-13T16:58:11.587 INFO:scrape:Failure: Test failure: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction) 2026-02-13T16:58:11.587 INFO:scrape:1 jobs: ['48154'] 2026-02-13T16:58:11.588 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rpm_latest}', 'fs/volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/volumes/{0-use-snaprealm/{global}', 'test/misc}}'] 2026-02-13T16:58:11.588 INFO:scrape: 2026-02-13T16:58:11.588 INFO:scrape:Failure: Test failure: test_progress_event_unique_per_process (tasks.cephfs.test_data_scan.TestDataScan.test_progress_event_unique_per_process) 2026-02-13T16:58:11.588 INFO:scrape:1 jobs: ['47978'] 2026-02-13T16:58:11.588 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rpm_latest}', 'fs/functional/{0-use-snaprealm/{subvol}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'pg_health}', 'rpm_latest}', 'tasks/data-scan}'] 2026-02-13T16:58:11.588 INFO:scrape: 2026-02-13T16:58:11.588 INFO:scrape:Failure: Command failed (workunit test suites/pjd.sh) on trial086 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' 2026-02-13T16:58:11.588 INFO:scrape:1 jobs: ['48021'] 2026-02-13T16:58:11.588 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/fixed-2-ucephfs', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/permission/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount/fuse', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'tasks/cfuse_workunit_suites_pjd}'] 2026-02-13T16:58:11.588 INFO:scrape: 2026-02-13T16:58:11.588 INFO:scrape:Failure: valgrind error: MismatchedFree operator delete[](void*, unsigned long, std::align_val_t) UnknownInlinedFun UnknownInlinedFun 2026-02-13T16:58:11.588 INFO:scrape:2 jobs: ['48007', '48101'] 2026-02-13T16:58:11.588 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rpm_latest}', 'fs/verify/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mon-debug', 'mount/fuse', 'osd}', 'overrides/{ignorelist_health', 'pg_health', 'session_timeout}', 'tasks/dbench', 'validater/valgrind}'] 2026-02-13T16:58:11.588 INFO:scrape:suites union: ['1-ceph', '2-logrotate', '3-kernel}', 'clusters/1a5s-mds-1c-client', 'conf/{client', 'distro/{rpm_latest}', 'fs/verify/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mon-debug', 'mount/fuse', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'pg_health', 'ranks/3', 'ranks/5', 'session_timeout}', 'tasks/dbench', 'validater/valgrind}'] 2026-02-13T16:58:11.588 INFO:scrape: 2026-02-13T16:58:11.588 INFO:scrape:Failure: Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.TestJournalMigration.test_journal_migration) 2026-02-13T16:58:11.588 INFO:scrape:1 jobs: ['48094'] 2026-02-13T16:58:11.588 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'begin/{0-install', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rpm_latest}', 'fs/functional/{0-use-snaprealm/{subvol}', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'pg_health}', 'rpm_latest}', 'tasks/test_journal_migration}'] 2026-02-13T16:58:11.588 INFO:scrape: 2026-02-13T16:58:11.588 INFO:scrape:Failure: Command failed (workunit test suites/ffsb.sh) on trial039 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2026-02-13T16:58:11.588 INFO:scrape:1 jobs: ['48093'] 2026-02-13T16:58:11.588 INFO:scrape:suites: ['0-subvolume/{with-quota}', '0-use-snaprealm/{global}', '1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/no', '3-kernel}', '3-snaps/yes', '4-flush/no', '5-quiesce/with-quiesce', '6-workunit/suites/ffsb}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-distro', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/secure', 'n/3', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/testing/k-testing', 'pg_health', 'ranks/multi/{balancer/random', 'replication/default}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}', 'wsync/no}'] 2026-02-13T16:58:11.588 INFO:scrape: 2026-02-13T16:58:11.588 INFO:scrape:Failure: Test failure: test_async_io_fio (tasks.cephfs.test_nfs.TestNFS.test_async_io_fio) 2026-02-13T16:58:11.588 INFO:scrape:1 jobs: ['47990'] 2026-02-13T16:58:11.588 INFO:scrape:suites: ['conf/{client', 'fs/nfs/{cluster/{1-node}', 'ignorelist_health', 'mds', 'mgr', 'mon', 'nfs-version/{unset}', 'osd}', 'overrides/{ignore_mgr_down', 'pg_health}', 'supported-random-distros$/{rpm_latest}', 'tasks/nfs}'] 2026-02-13T16:58:11.588 INFO:scrape: 2026-02-13T16:58:11.588 INFO:scrape:Failure: Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.test_mirroring.TestMirroring.test_cephfs_mirror_restart_sync_on_blocklist) 2026-02-13T16:58:11.589 INFO:scrape:1 jobs: ['47985'] 2026-02-13T16:58:11.589 INFO:scrape:suites: ['1-ceph', '2-logrotate', '3-kernel}', 'cephfs-mirror/one-per-cluster', 'clients/{mirror}', 'cluster/{1-node}', 'conf/{client', 'fs/mirror/{begin/{0-install', 'mds', 'mgr', 'mon', 'mount/fuse', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'pg_health}', 'supported-random-distros$/{rpm_latest}', 'tasks/mirror}'] 2026-02-13T16:58:11.589 INFO:scrape: 2026-02-13T16:58:11.589 INFO:scrape:Failure: Command failed (workunit test kernel_untar_build.sh) on trial026 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 5h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' 2026-02-13T16:58:11.589 INFO:scrape:2 jobs: ['48131', '48079'] 2026-02-13T16:58:11.589 INFO:scrape:suites intersection: ['0-use-snaprealm/{subvol}', '1-cephadm', '1-check-counter', '2-logrotate', '3-kernel}', '3-snaps/yes', '4-flush/no', '6-workunit/kernel_untar_build}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'frag', 'fs/workload/{0-distro', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'ms-die-on-skipped}}', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'pg_health', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}'] 2026-02-13T16:58:11.589 INFO:scrape:suites union: ['0-subvolume/{with-no-extra-options}', '0-subvolume/{with-quota}', '0-use-snaprealm/{subvol}', '1-cephadm', '1-check-counter', '2-logrotate', '2-scrub/no', '2-scrub/yes', '3-kernel}', '3-snaps/yes', '4-flush/no', '5-quiesce/no', '5-quiesce/with-quiesce', '6-workunit/kernel_untar_build}}', 'begin/{0-install', 'clusters/1a11s-mds-1c-client-3node', 'conf/{client', 'export-check', 'frag', 'fs/workload/{0-distro', 'ignorelist_health', 'ignorelist_wrongly_marked_down', 'mds', 'mgr', 'mon', 'mount', 'mount/kclient/{base/{mount-syntax/{v1}', 'mount/kclient/{base/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'ms_mode/legacy', 'ms_mode/secure', 'n/5', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'omap_limit/10', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{cephsqlite-timeout', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'pg_health', 'ranks/1', 'ranks/multi/{balancer/automatic', 'replication/default}', 'rpm_latest}', 'session_timeout}', 'standby-replay', 'tasks/{0-fscrypt/{with-fscrypt-dummy}', 'wsync/no}', 'wsync/yes}'] 2026-02-13T16:58:11.589 INFO:scrape: 2026-02-13T16:58:11.594 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial ================================================================= info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/ logs: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/ failed: 143 dead: 17 running: 0 waiting: 2 queued: 0 passed: 79 Fail ================================================================= [47949] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/iogen}} ----------------------------------------------------------------- time: 00:11:58 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47949/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47949/ "2026-02-13T02:05:37.632156+0000 mon.a (mon.0) 307 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47951] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:22:44 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47951/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47951/ "2026-02-13T02:08:13.135744+0000 mon.a (mon.0) 375 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47952] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/3 tasks/fsstress validater/lockdep} ----------------------------------------------------------------- time: 00:08:34 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47952/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47952/ Command failed on trial168 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback' [47954] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:13:11 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47954/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47954/ "2026-02-13T02:06:54.492724+0000 mon.a (mon.0) 305 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47959] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/direct_io}} ----------------------------------------------------------------- time: 00:13:43 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47959/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47959/ "2026-02-13T02:12:14.675447+0000 mon.a (mon.0) 379 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47960] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} ----------------------------------------------------------------- time: 00:04:26 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47960/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47960/ Command failed (workunit test suites/iozone.sh) on trial077 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [47961] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/tentacle 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:02:29 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47961/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47961/ [47962] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/fs/misc}} ----------------------------------------------------------------- time: 00:12:17 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47962/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47962/ "2026-02-13T02:12:37.166913+0000 mon.a (mon.0) 315 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47964] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} ----------------------------------------------------------------- time: 00:27:08 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47964/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47964/ "2026-02-13T02:17:03.543792+0000 mon.a (mon.0) 407 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47965] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/with-quiesce 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:04:01 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47965/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47965/ Command failed (workunit test suites/iozone.sh) on trial031 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [47968] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/postgres}} ----------------------------------------------------------------- time: 00:11:25 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47968/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47968/ "2026-02-13T02:17:18.505391+0000 mon.a (mon.0) 309 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47971] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:13:03 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47971/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47971/ "2026-02-13T02:20:40.028769+0000 mon.a (mon.0) 406 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47973] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} ----------------------------------------------------------------- time: 00:12:07 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47973/ "2026-02-13T02:20:31.306060+0000 mon.a (mon.0) 301 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47974] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/damage} ----------------------------------------------------------------- time: 00:04:32 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47974/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47974/ Fuse mount failed to populate/sys/ after 31 seconds [47975] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:13:23 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47975/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47975/ "2026-02-13T02:21:57.229718+0000 mon.a (mon.0) 341 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47977] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/fs/norstats}} ----------------------------------------------------------------- time: 00:11:29 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47977/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47977/ "2026-02-13T02:20:39.778454+0000 mon.a (mon.0) 298 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47978] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/data-scan} ----------------------------------------------------------------- time: 00:15:49 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47978/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47978/ Test failure: test_progress_event_unique_per_process (tasks.cephfs.test_dat a_scan.TestDataScan.test_progress_event_unique_per_process) [47982] fs/cephadm/multivolume/{0-start 1-mount 2-workload/dbench conf/{client mds mgr mon osd} distro/single-container-host overrides/{ignorelist_health pg_health}} ----------------------------------------------------------------- time: 00:47:35 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47982/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47982/ "2026-02-13T02:22:45.753103+0000 mon.trial127 (mon.0) 375 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47984] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client} ----------------------------------------------------------------- time: 00:06:09 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47984/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47984/ Command failed (workunit test client/test_oc_disabled.sh) on trial091 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/client/test_oc_disabled.sh' [47985] fs/mirror/{begin/{0-install 1-ceph 2-logrotate 3-kernel} cephfs-mirror/one-per-cluster clients/{mirror} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{rpm_latest} tasks/mirror} ----------------------------------------------------------------- time: 01:14:02 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47985/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47985/ Test failure: test_cephfs_mirror_restart_sync_on_blocklist (tasks.cephfs.te st_mirroring.TestMirroring.test_cephfs_mirror_restart_sync_on_blocklist) [47988] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:05:43 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47988/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47988/ Fuse mount failed to populate/sys/ after 31 seconds [47989] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/multifs-auth} ----------------------------------------------------------------- time: 00:05:10 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47989/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47989/ Fuse mount failed to populate/sys/ after 31 seconds [47990] fs/nfs/{cluster/{1-node} conf/{client mds mgr mon osd} nfs-version/{unset} overrides/{ignore_mgr_down ignorelist_health pg_health} supported-random-distros$/{rpm_latest} tasks/nfs} ----------------------------------------------------------------- time: 00:04:34 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47990/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47990/ Test failure: test_async_io_fio (tasks.cephfs.test_nfs.TestNFS.test_async_io_fio) [47991] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_misc} ----------------------------------------------------------------- time: 00:21:28 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47991/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47991/ Command failed (workunit test fs/misc/chmod.sh) on trial082 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh' [47993] fs/snaps/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:13:23 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47993/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47993/ Command failed (workunit test fs/snaps/untar_snap_rm.sh) on trial088 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' [47995] fs/valgrind/{begin/{0-install 1-ceph 2-logrotate 3-kernel} debug mirror/{cephfs-mirror/one-per-cluster clients/mirror cluster/1-node mount/fuse overrides/{ignorelist_health pg_health} tasks/mirror} rpm_latest} ----------------------------------------------------------------- time: 00:08:07 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47995/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47995/ Test failure: test_add_ancestor_and_child_directory (tasks.cephfs.test_mirr oring.TestMirroring.test_add_ancestor_and_child_directory) [47997] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:13:29 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47997/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47997/ "2026-02-13T02:29:24.317197+0000 mon.a (mon.0) 376 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [47998] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} ----------------------------------------------------------------- time: 00:04:45 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47998/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47998/ Fuse mount failed to populate/sys/ after 31 seconds [48000] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/fsx}} ----------------------------------------------------------------- time: 00:12:39 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48000/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48000/ "2026-02-13T02:32:21.355181+0000 mon.a (mon.0) 306 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48001] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/forward-scrub} ----------------------------------------------------------------- time: 00:03:44 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48001/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48001/ Fuse mount failed to populate/sys/ after 31 seconds [48002] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} ----------------------------------------------------------------- time: 00:23:39 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48002/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48002/ "2026-02-13T02:35:43.420499+0000 mon.a (mon.0) 343 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48005] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/fs/test_o_trunc}} ----------------------------------------------------------------- time: 00:21:22 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48005/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48005/ "2026-02-13T02:34:13.908439+0000 mon.a (mon.0) 313 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48007] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/5 tasks/dbench validater/valgrind} ----------------------------------------------------------------- time: 00:18:46 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48007/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48007/ valgrind error: MismatchedFree operator delete[](void*, unsigned long, std::align_val_t) UnknownInlinedFun UnknownInlinedFun [48008] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-iozone}} ----------------------------------------------------------------- time: 00:05:28 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48008/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48008/ Command failed on trial117 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback' [48009] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iogen}} ----------------------------------------------------------------- time: 00:13:10 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48009/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48009/ "2026-02-13T02:39:58.483220+0000 mon.a (mon.0) 406 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48011] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/journal-repair} ----------------------------------------------------------------- time: 00:04:46 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48011/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48011/ Fuse mount failed to populate/sys/ after 31 seconds [48012] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:12:18 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48012/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48012/ "2026-02-13T02:39:10.068406+0000 mon.a (mon.0) 301 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48013] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:15:14 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48013/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48013/ Command failed on trial034 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f117fb9a-0884-11f1-a293-d404e6e7d460 -e sha1=8f8d38294e4db8750b4b1649f62ce37caa63080f -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [48014] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:13:29 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48014/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48014/ "2026-02-13T02:42:13.039346+0000 mon.a (mon.0) 409 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48018] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/direct_io}} ----------------------------------------------------------------- time: 00:11:30 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48018/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48018/ "2026-02-13T02:39:55.265388+0000 mon.a (mon.0) 309 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48020] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:10:35 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48020/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48020/ reached maximum tries (50) after waiting for 300 seconds [48021] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:19:37 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48021/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48021/ Command failed (workunit test suites/pjd.sh) on trial086 with status 3: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/pjd.sh' [48022] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/misc}} ----------------------------------------------------------------- time: 00:13:58 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48022/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48022/ "2026-02-13T02:44:10.611276+0000 mon.a (mon.0) 321 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48025] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_trivial_sync}} ----------------------------------------------------------------- time: 00:08:28 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48025/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48025/ Command failed on trial096 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.a=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback' [48026] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/kernel_untar_build}} ----------------------------------------------------------------- time: 00:12:11 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48026/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48026/ "2026-02-13T02:46:21.225709+0000 mon.a (mon.0) 302 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48027] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} ----------------------------------------------------------------- time: 00:13:58 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48027/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48027/ "2026-02-13T02:48:49.253166+0000 mon.a (mon.0) 376 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48028] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/tentacle 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/fsstress}} ----------------------------------------------------------------- time: 00:01:52 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48028/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48028/ Command failed on trial039 with status 1: 'sudo yum -y install ceph-osd- classic' [48030] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{rpm_latest} kclient-overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} ----------------------------------------------------------------- time: 03:11:34 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48030/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48030/ Command failed (workunit test suites/iozone.sh) on trial043 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [48031] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/ior-shared-file} ----------------------------------------------------------------- time: 00:04:30 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48031/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48031/ Fuse mount failed to populate/sys/ after 31 seconds [48033] fs/snaps/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:13:46 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48033/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48033/ Command failed (workunit test fs/snaps/untar_snap_rm.sh) on trial077 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' [48035] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:10:59 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48035/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48035/ Command failed (workunit test suites/blogbench.sh) on trial070 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh' [48037] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/tentacle 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:02:30 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48037/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48037/ [48038] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} ----------------------------------------------------------------- time: 00:13:35 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48038/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48038/ "2026-02-13T02:57:33.693829+0000 mon.a (mon.0) 378 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48040] fs/bugs/multifs_mdsauthcaps/{clusters/1node-cluster-1node-client conf/{client mds mgr mon osd} distro/{rpm_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/{0-install 1-mount/fuse 2-test_admin_squid_client}} ----------------------------------------------------------------- time: 00:01:03 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48040/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48040/ Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=rocky%2F10%2Fx86_64&sha1=0ec eb0defba60152a8182f7bd87d164b639885b8 [48041] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-pjd}} ----------------------------------------------------------------- time: 00:05:40 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48041/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48041/ Command failed (workunit test fs/fscrypt.sh) on trial194 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh unlocked pjd' [48043] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:12:18 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48043/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48043/ "2026-02-13T02:59:05.641822+0000 mon.a (mon.0) 301 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48044] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/openfiletable} ----------------------------------------------------------------- time: 00:05:04 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48044/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48044/ Fuse mount failed to populate/sys/ after 31 seconds [48046] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/norstats}} ----------------------------------------------------------------- time: 00:23:24 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48046/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48046/ "2026-02-13T03:03:55.040234+0000 mon.a (mon.0) 408 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48047] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{global} overrides test/clone-progress}} ----------------------------------------------------------------- time: 00:06:31 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48047/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48047/ Test failure: test_clone_after_subvol_is_removed (tasks.cephfs.volumes.test _clone_stats.TestCloneProgressReporter.test_clone_after_subvol_is_removed) [48049] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/with-quiesce 2-workunit/fs/trivial_sync}} ----------------------------------------------------------------- time: 03:05:57 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48049/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48049/ Command failed (workunit test fs/misc/trivial_sync.sh) on trial011 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh' [48050] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:12:04 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48050/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48050/ "2026-02-13T03:00:46.196249+0000 mon.a (mon.0) 307 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48052] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsx}} ----------------------------------------------------------------- time: 00:24:15 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48052/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48052/ "2026-02-13T03:04:42.205332+0000 mon.a (mon.0) 409 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48054] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_misc} ----------------------------------------------------------------- time: 00:19:31 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48054/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48054/ Command failed (workunit test fs/misc/chmod.sh) on trial089 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh' [48056] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/iozone}} ----------------------------------------------------------------- time: 00:04:07 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48056/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48056/ Command failed (workunit test suites/iozone.sh) on trial071 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/iozone.sh' [48057] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsync-tester}} ----------------------------------------------------------------- time: 00:12:23 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48057/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48057/ "2026-02-13T03:05:46.513278+0000 mon.a (mon.0) 310 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48059] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} ----------------------------------------------------------------- time: 00:13:43 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48059/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48059/ "2026-02-13T03:07:33.619354+0000 mon.a (mon.0) 376 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48060] fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} kernel no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/tentacle 1-upgrade}} ----------------------------------------------------------------- time: 00:02:20 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48060/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48060/ Command failed on trial169 with status 1: 'sudo yum -y install ceph-osd- classic' [48062] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/iogen}} ----------------------------------------------------------------- time: 00:11:32 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48062/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48062/ "2026-02-13T03:07:22.130189+0000 mon.a (mon.0) 300 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48063] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/exports 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:03:52 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48063/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48063/ Command failed (workunit test suites/iozone.sh) on trial086 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [48064] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:14:09 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48064/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48064/ Command failed on trial138 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8223b82f-0889-11f1-b4d7-d404e6e7d460 -e sha1=8f8d38294e4db8750b4b1649f62ce37caa63080f -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [48065] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:23:04 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48065/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48065/ "2026-02-13T03:12:53.335949+0000 mon.a (mon.0) 375 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48067] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:12:35 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48067/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48067/ "2026-02-13T03:13:04.297986+0000 mon.a (mon.0) 300 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48068] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:05:54 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48068/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48068/ error during thrashing [48069] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_blogbench traceless/50pc} ----------------------------------------------------------------- time: 00:05:49 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48069/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48069/ Command failed (workunit test suites/blogbench.sh) on trial015 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh' [48072] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/mdtest} ----------------------------------------------------------------- time: 00:05:07 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48072/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48072/ Fuse mount failed to populate/sys/ after 31 seconds [48074] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/direct_io}} ----------------------------------------------------------------- time: 00:23:42 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48074/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48074/ "2026-02-13T03:18:27.758662+0000 mon.a (mon.0) 403 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48075] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/fs/misc}} ----------------------------------------------------------------- time: 00:18:38 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48075/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48075/ "2026-02-13T03:17:17.277145+0000 mon.a (mon.0) 304 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48077] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:12:46 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48077/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48077/ Command failed (workunit test fs/snaps/untar_snap_rm.sh) on trial104 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clo ne.client.1/qa/workunits/fs/snaps/untar_snap_rm.sh' [48079] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} ----------------------------------------------------------------- time: 00:26:23 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48079/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48079/ Command failed (workunit test kernel_untar_build.sh) on trial121 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 5h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' [48084] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/postgres}} ----------------------------------------------------------------- time: 00:11:53 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48084/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48084/ "2026-02-13T03:24:14.841593+0000 mon.a (mon.0) 302 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48085] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snapshots} ----------------------------------------------------------------- time: 00:08:07 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48085/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48085/ Command failed on trial075 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph. conf,norbytes,name=0,mds_namespace=cephfs' [48086] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:13:28 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48086/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48086/ "2026-02-13T03:28:01.429379+0000 mon.a (mon.0) 407 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48090] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} ----------------------------------------------------------------- time: 00:45:19 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48090/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48090/ "2026-02-13T03:27:26.311347+0000 mon.a (mon.0) 322 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48091] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/strays} ----------------------------------------------------------------- time: 00:03:36 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48091/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48091/ Fuse mount failed to populate/sys/ after 31 seconds [48092] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/tentacle 1-mount/mount/fuse 2-clients/fuse-upgrade 3-workload/stress_tests/blogbench}} ----------------------------------------------------------------- time: 00:01:57 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48092/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48092/ Command failed on trial089 with status 1: 'sudo yum -y install ceph-osd- classic' [48093] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:25:17 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48093/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48093/ Command failed (workunit test suites/ffsb.sh) on trial039 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' [48094] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/test_journal_migration} ----------------------------------------------------------------- time: 00:05:32 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48094/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48094/ Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.T estJournalMigration.test_journal_migration) [48095] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/fs/norstats}} ----------------------------------------------------------------- time: 00:11:50 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48095/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48095/ "2026-02-13T03:32:27.488029+0000 mon.a (mon.0) 302 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48096] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 02:13:01 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48096/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48096/ reached maximum tries (50) after waiting for 300 seconds [48099] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:13:27 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48099/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48099/ "2026-02-13T03:36:04.027787+0000 mon.a (mon.0) 328 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48100] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/test_snap_schedule/{overrides/{v2} snap-schedule}} ----------------------------------------------------------------- time: 00:04:55 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48100/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48100/ Fuse mount failed to populate/sys/ after 31 seconds [48101] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/3 tasks/dbench validater/valgrind} ----------------------------------------------------------------- time: 00:18:34 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48101/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48101/ valgrind error: MismatchedFree operator delete[](void*, unsigned long, std::align_val_t) UnknownInlinedFun UnknownInlinedFun [48102] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/fsx}} ----------------------------------------------------------------- time: 00:12:19 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48102/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48102/ "2026-02-13T03:38:16.154090+0000 mon.a (mon.0) 306 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48103] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:05:52 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48103/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48103/ error during thrashing [48106] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/tentacle 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:02:03 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48106/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48106/ [48108] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/fs/test_o_trunc}} ----------------------------------------------------------------- time: 00:12:12 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48108/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48108/ "2026-02-13T03:43:39.022574+0000 mon.a (mon.0) 309 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48109] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:10:16 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48109/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48109/ reached maximum tries (50) after waiting for 300 seconds [48110] fs/cephadm/renamevolume/{0-start 1-rename conf/{client mds mgr mon osd} distro/single-container-host overrides/{ignorelist_health pg_health}} ----------------------------------------------------------------- time: 00:10:01 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48110/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48110/ "2026-02-13T03:44:10.655649+0000 mon.trial026 (mon.0) 377 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48111] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{rpm_latest} kclient-overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/fscrypt-kbuild} ----------------------------------------------------------------- time: 04:12:25 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48111/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48111/ Command failed (workunit test fs/fscrypt_kclient_fuse.sh) on trial126 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/fscrypt-fuse && cd -- /home/ubuntu/cephtest/mnt.0/fscrypt-fuse && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 4h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/fscrypt_kclient_fuse.sh fuse' [48112] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cephfs_misc_tests} ----------------------------------------------------------------- time: 00:04:27 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48112/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48112/ Fuse mount failed to populate/sys/ after 31 seconds [48115] fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} ----------------------------------------------------------------- time: 00:25:38 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48115/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48115/ Test failure: test_get_doesnt_create_dir (tasks.cephfs.test_cephfs_shell.TestGetAndPut.test_get_doesnt_create_dir) [48119] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/iogen}} ----------------------------------------------------------------- time: 00:23:45 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48119/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48119/ "2026-02-13T03:53:58.102698+0000 mon.a (mon.0) 390 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48121] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:11:29 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48121/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48121/ "2026-02-13T03:51:44.119363+0000 mon.a (mon.0) 311 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48125] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:13:40 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48125/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48125/ "2026-02-13T03:58:54.720761+0000 mon.a (mon.0) 372 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48127] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/direct_io}} ----------------------------------------------------------------- time: 00:12:02 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48127/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48127/ "2026-02-13T03:58:34.567358+0000 mon.a (mon.0) 414 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48129] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/misc}} ----------------------------------------------------------------- time: 00:12:53 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48129/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48129/ "2026-02-13T04:01:19.914628+0000 mon.a (mon.0) 301 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48131] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/kernel_untar_build}} ----------------------------------------------------------------- time: 00:27:50 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48131/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48131/ Command failed (workunit test kernel_untar_build.sh) on trial026 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 5h /home/ubuntu/cephtest/clone.client.0/qa/workunits/kernel_untar_build.sh' [48133] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:08:23 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48133/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48133/ Command failed on trial104 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.a=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback' [48135] fs/bugs/multifs_mdsauthcaps/{clusters/1node-cluster-1node-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/{0-install 1-mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} 2-test_admin_squid_client}} ----------------------------------------------------------------- time: 00:01:05 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48135/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48135/ Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=rocky%2F10%2Fx86_64&sha1=0ec eb0defba60152a8182f7bd87d164b639885b8 [48136] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-ffsb}} ----------------------------------------------------------------- time: 00:24:33 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48136/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48136/ Command failed (workunit test fs/fscrypt.sh) on trial131 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh unlocked ffsb' [48138] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} ----------------------------------------------------------------- time: 00:10:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48138/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48138/ Command failed on trial077 with status 1: 'sudo TESTDIR=/home/ubuntu/cephtest bash -c \'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"\'' [48139] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/blogbench}} ----------------------------------------------------------------- time: 00:25:36 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48139/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48139/ Command failed (workunit test suites/blogbench.sh) on trial089 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh' [48141] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/suites/iozone}} ----------------------------------------------------------------- time: 03:04:36 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48141/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48141/ Command failed (workunit test suites/iozone.sh) on trial096 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [48143] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_misc} ----------------------------------------------------------------- time: 00:03:46 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48143/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48143/ Fuse mount failed to populate/sys/ after 31 seconds [48144] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} ----------------------------------------------------------------- time: 00:13:27 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48144/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48144/ "2026-02-13T04:19:48.924046+0000 mon.a (mon.0) 302 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48146] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/ffsb}} ----------------------------------------------------------------- time: 00:23:50 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48146/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48146/ "2026-02-13T04:19:33.826384+0000 mon.a (mon.0) 409 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48147] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/iozone}} ----------------------------------------------------------------- time: 00:04:40 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48147/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48147/ Command failed (workunit test suites/iozone.sh) on trial134 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh' [48148] fs/verify/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health session_timeout} ranks/5 tasks/fsstress validater/lockdep} ----------------------------------------------------------------- time: 00:08:37 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48148/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48148/ Command failed on trial104 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback' [48149] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/norstats}} ----------------------------------------------------------------- time: 00:13:09 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48149/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48149/ "2026-02-13T04:22:56.477037+0000 mon.a (mon.0) 306 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48150] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/auto-repair} ----------------------------------------------------------------- time: 00:03:47 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48150/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48150/ Fuse mount failed to populate/sys/ after 31 seconds [48151] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/ior-shared-file} ----------------------------------------------------------------- time: 00:04:55 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48151/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48151/ Fuse mount failed to populate/sys/ after 31 seconds [48153] fs/snaps/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:08:37 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48153/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48153/ Command failed (workunit test fs/snaps/untar_snap_rm.sh) on trial025 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' [48154] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{global} overrides test/misc}} ----------------------------------------------------------------- time: 00:07:15 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48154/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48154/ Test failure: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction) [48155] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/fsstress}} ----------------------------------------------------------------- time: 00:12:01 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48155/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48155/ "2026-02-13T04:28:44.298715+0000 mon.a (mon.0) 413 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48157] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:15:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48157/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48157/ Command failed on trial070 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1541a575-0895-11f1-ba5f-d404e6e7d460 -e sha1=8f8d38294e4db8750b4b1649f62ce37caa63080f -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' [48159] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} ----------------------------------------------------------------- time: 00:13:42 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48159/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48159/ "2026-02-13T04:38:06.369853+0000 mon.a (mon.0) 304 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48161] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-no-extra-options} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/fsync-tester}} ----------------------------------------------------------------- time: 00:11:29 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48161/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48161/ "2026-02-13T04:35:19.164264+0000 mon.a (mon.0) 409 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48163] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_snaptests}} ----------------------------------------------------------------- time: 00:13:24 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48163/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48163/ Command failed (workunit test fs/snaps/untar_snap_rm.sh) on trial118 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' [48164] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} ----------------------------------------------------------------- time: 00:13:32 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48164/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48164/ "2026-02-13T04:41:09.485232+0000 mon.a (mon.0) 321 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48166] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-iozone}} ----------------------------------------------------------------- time: 00:04:13 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48166/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48166/ Command failed (workunit test fs/fscrypt.sh) on trial189 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/fscrypt.sh none iozone' [48167] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iogen}} ----------------------------------------------------------------- time: 00:30:23 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48167/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48167/ "2026-02-13T04:43:01.963743+0000 mon.a (mon.0) 407 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48169] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} ----------------------------------------------------------------- time: 00:13:26 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48169/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48169/ "2026-02-13T04:45:31.531002+0000 mon.a (mon.0) 303 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48170] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:10:24 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48170/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48170/ reached maximum tries (50) after waiting for 300 seconds [48173] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/pjd}} ----------------------------------------------------------------- time: 00:11:33 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48173/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48173/ "2026-02-13T04:51:18.003706+0000 mon.a (mon.0) 409 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48174] fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} kernel no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/tentacle 1-upgrade}} ----------------------------------------------------------------- time: 00:01:50 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48174/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48174/ Command failed on trial177 with status 1: 'sudo yum -y install ceph-osd- classic' [48176] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/direct_io}} ----------------------------------------------------------------- time: 00:13:13 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48176/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48176/ "2026-02-13T04:54:51.639907+0000 mon.a (mon.0) 302 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48178] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/fs/misc}} ----------------------------------------------------------------- time: 00:12:19 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48178/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48178/ "2026-02-13T04:53:40.181385+0000 mon.a (mon.0) 411 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48180] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/tentacle 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:02:10 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48180/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48180/ [48183] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/kernel_untar_build}} ----------------------------------------------------------------- time: 00:13:08 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48183/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48183/ "2026-02-13T04:59:31.164489+0000 mon.a (mon.0) 307 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48184] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/postgres}} ----------------------------------------------------------------- time: 00:11:22 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48184/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48184/ "2026-02-13T04:58:53.242419+0000 mon.a (mon.0) 407 : cluster [WRN] Health check failed: Failed to apply 1 service(s): osd.default (CEPHADM_APPLY_SPEC_FAIL)" in cluster log [48185] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/data-scan} ----------------------------------------------------------------- time: 00:04:53 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48185/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48185/ Fuse mount failed to populate/sys/ after 31 seconds [48186] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 03:11:56 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48186/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48186/ Command failed (workunit test suites/blogbench.sh) on trial081 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=cfcf307502511ba51c39a8b9085c2fcde505d421 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh' [48187] fs/multiclient/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-3-client conf/{client mds mgr mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/mdtest} ----------------------------------------------------------------- time: 00:04:22 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48187/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48187/ Fuse mount failed to populate/sys/ after 31 seconds Dead ================================================================= [47948] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/ffsb}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47948/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47948/ hit max job timeout [47972] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-recovery} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47972/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47972/ hit max job timeout [47987] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/fscrypt-kbuild} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47987/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47987/ hit max job timeout [47996] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{global} overrides test/async}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47996/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47996/ hit max job timeout [48004] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48004/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48004/ hit max job timeout [48010] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48010/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48010/ Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds [48051] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/quiesce} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48051/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48051/ hit max job timeout [48058] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/quota} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48058/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48058/ hit max job timeout [48066] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/referent} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48066/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48066/ hit max job timeout [48070] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-common}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48070/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48070/ hit max job timeout [48071] fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu/{overrides ubuntu_latest}} kclient-overrides/{distro/testing/k-testing ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_untarbuild_blogbench} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48071/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48071/ hit max job timeout [48078] fs/upgrade/upgraded_client/{bluestore-bitmap centos_9.stream clusters/{1-mds-1-client-micro} conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-install/squid 1-mount/mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} 2-clients/kclient 3-workload/new_ops/{newops}}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48078/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48078/ hit max job timeout [48105] fs/workload/{0-distro begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{subvol} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48105/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48105/ Error reimaging machines: reached maximum tries (100) after waiting for 600 seconds [48113] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/failover} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48113/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48113/ hit max job timeout [48116] fs/snaps/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/workunit/snaps} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48116/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48116/ hit max job timeout [48123] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/xfstests-dev} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48123/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48123/ hit max job timeout [48130] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/acls} ----------------------------------------------------------------- time: 00:00:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48130/ log: https://qa-proxy.ceph.com/teuthology/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48130/ hit max job timeout Waiting ================================================================= [48016] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48016/ [48017] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{subvol} overrides test/basic}} info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48017/ Pass ================================================================= [47950] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/backtrace} time: 00:04:16 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47950/ [47953] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cap-flush} time: 00:03:58 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47953/ [47955] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress} time: 00:05:18 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47955/ [47956] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_misc} time: 00:04:36 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47956/ [47957] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/tentacle 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:09:38 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47957/ [47958] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{global} overrides test/subvolume-snapshots-visibility}} time: 00:18:53 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47958/ [47963] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-limits} time: 00:14:30 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47963/ [47966] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-readahead} time: 00:03:44 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47966/ [47967] fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate 3-kernel} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn pg_health} tasks/{1-tests/fscrypt-ffsb}} time: 00:31:01 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47967/ [47969] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:28:50 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47969/ [47970] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-compat_client/yes}} time: 00:08:36 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47970/ [47976] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/exports 2-workunit/suites/pjd}} time: 00:14:47 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47976/ [47979] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:12:36 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47979/ [47980] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress} time: 00:04:44 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47980/ [47981] fs/bugs/client_trim_caps/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/small-cluster conf/{client mds mgr mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} rpm_latest tasks/trim-i24137} time: 00:04:16 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47981/ [47983] fs/full/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-node-4-mds-1-osd conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/mgr-osd-full} time: 00:10:14 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47983/ [47986] fs/mirror-ha/{begin/{0-install 1-ceph 2-logrotate 3-kernel} cephfs-mirror/{1-volume-create-rm 2-three-per-cluster} clients/{mirror} cluster/{1-node} conf/{client mds mgr mon osd} objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distro$/{rpm_latest} workloads/cephfs-mirror-ha-workunit} time: 00:11:16 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47986/ [47992] fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/rpm_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} time: 00:27:09 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47992/ [47994] fs/top/{begin/{0-install 1-ceph 2-logrotate 3-kernel} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{rpm_latest} tasks/fstop} time: 00:05:05 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47994/ [47999] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-compat_client/no}} time: 00:10:38 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/47999/ [48003] fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} kernel no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/squid 1-upgrade}} time: 00:04:37 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48003/ [48006] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/fragment} time: 00:40:37 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48006/ [48015] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/mds-flush} time: 00:04:04 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48015/ [48019] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/mds-full} time: 00:29:50 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48019/ [48023] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress traceless/50pc} time: 00:05:01 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48023/ [48024] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:10:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48024/ [48029] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/metrics} time: 00:16:43 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48029/ [48032] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/failover} time: 01:30:09 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48032/ [48034] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} time: 00:05:43 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48034/ [48036] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/multimds_misc} time: 00:19:39 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48036/ [48039] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/ffsb}} time: 00:18:33 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48039/ [48042] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{rpm_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client_full} time: 00:05:53 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48042/ [48045] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/tentacle 1-client 2-upgrade 3-compat_client/yes}} time: 00:08:44 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48045/ [48048] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/pool-perm} time: 00:04:56 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48048/ [48053] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress} time: 00:04:40 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48053/ [48055] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/tentacle 1-client 2-upgrade 3-compat_client/no}} time: 00:11:24 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48055/ [48061] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/recovery-fs} time: 00:04:09 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48061/ [48073] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{global} overrides test/clone}} time: 00:43:46 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48073/ [48076] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/sessionmap} time: 00:08:28 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48076/ [48080] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snap_schedule_snapdir} time: 00:04:01 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48080/ [48081] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/tentacle 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:10:44 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48081/ [48082] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:04:53 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48082/ [48083] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:05:25 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48083/ [48087] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:13:15 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48087/ [48088] fs/bugs/snapshot_visibility_on_upgrade/{cluster/1-node conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/{0-install 1-test-subvolume-snapshot-default-visibility}} time: 00:04:44 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48088/ [48089] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{rpm_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/ino_release_cb} time: 00:04:13 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48089/ [48097] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{global} overrides test/finisher_per_module}} time: 00:03:43 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48097/ [48098] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-compat_client/yes}} time: 00:09:06 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48098/ [48104] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/truncate_delay} time: 00:04:20 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48104/ [48107] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/uninlining} time: 00:57:17 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48107/ [48114] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:04:46 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48114/ [48117] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/suites/fsstress}} time: 00:06:16 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48117/ [48118] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_dbench traceless/50pc} time: 00:39:36 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48118/ [48120] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-compat_client/no}} time: 00:10:24 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48120/ [48122] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_trivial_sync}} time: 00:04:47 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48122/ [48124] fs/upgrade/nofs/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} kernel no-mds-cluster overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn pg_health} tasks/{0-from/squid 1-upgrade}} time: 00:04:50 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48124/ [48126] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{global} overrides test/metrics}} time: 00:05:56 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48126/ [48128] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/workunit/quota} time: 00:03:54 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48128/ [48132] fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no kernel overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:12:52 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48132/ [48134] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/admin} time: 02:01:11 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48134/ [48137] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{ubuntu_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/libcephfs/{frag test}} time: 01:13:48 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48137/ [48140] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/alternate-pool} time: 00:04:48 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48140/ [48142] fs/32bits/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_fsstress} time: 00:05:19 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48142/ [48145] fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/squid 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:10:00 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48145/ [48152] fs/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug pg_health} tasks/multifs-auth} time: 00:24:44 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48152/ [48156] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/backtrace} time: 00:05:10 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48156/ [48158] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/yes prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/pjd}} time: 00:06:39 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48158/ [48160] fs/traceless/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_ffsb traceless/50pc} time: 00:16:28 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48160/ [48162] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/tentacle 1-client 2-upgrade 3-compat_client/yes}} time: 00:08:45 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48162/ [48165] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/charmap} time: 00:12:10 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48165/ [48168] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-limits} time: 00:24:04 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48168/ [48171] fs/permission/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/fixed-2-ucephfs conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down pg_health} tasks/cfuse_workunit_suites_pjd} time: 00:04:39 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48171/ [48172] fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_9.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} kernel overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-from/tentacle 1-client 2-upgrade 3-compat_client/no}} time: 00:11:32 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48172/ [48175] fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{0-use-snaprealm/{global} overrides test/snapshot}} time: 00:28:34 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48175/ [48177] fs/functional/{0-use-snaprealm/{subvol} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-recovery} time: 00:43:31 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48177/ [48179] fs/libcephfs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/{rpm_latest} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/libcephfs_python} time: 00:07:29 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48179/ [48181] fs/functional/{0-use-snaprealm/{global} begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rpm_latest} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/damage} time: 00:13:17 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48181/ [48182] fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_entire_dirfrags/no prefetch_oft_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/with-quiesce 2-workunit/suites/ffsb}} time: 00:23:14 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48182/ [48188] fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate 3-kernel} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{rpm_latest} mount/fuse msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs pg_health session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} time: 00:05:58 info: https://pulpito.ceph.com/pdonnell-2026-02-13_01:53:26-fs-wip-pdonnell-testing-20260210.171210-distro-default-trial/48188/ 2026-02-13T16:58:11.604 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/results.py", line 33, in main results(args['--archive-dir'], args['--name'], args['--email'], File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/results.py", line 78, in results email_results( File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/results.py", line 95, in email_results smtp = smtplib.SMTP('localhost') ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/smtplib.py", line 255, in __init__ (code, msg) = self.connect(host, port) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/smtplib.py", line 341, in connect self.sock = self._get_socket(host, port, self.timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/smtplib.py", line 312, in _get_socket return socket.create_connection((host, port), timeout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/gevent/socket.py", line 115, in create_connection sock.connect(sa) File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 590, in connect self._internal_connect(address) File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/virtualenv/lib/python3.12/site-packages/gevent/_socketcommon.py", line 634, in _internal_connect raise _SocketError(err, strerror(err)) ConnectionRefusedError: [Errno 111] Connection refused