Found 71 jobs
Found 8 distinct failure reasons
Failure: The WatchDog barked due to Daemon ceph.osd.0 has failed
6 jobs: ['68105', '68085', '68079', '68101', '68109', '68072']
suites intersection: ['0-size-min-size-overrides/3-size-2-min-size', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'thrashosds-health']
suites union: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'thrashers/{default}', 'thrashers/{pggrow}', 'thrashosds-health', 'workloads/pool-snaps-few-objects}', 'workloads/small-objects-balanced}', 'workloads/small-objects-localized}', 'workloads/snaps-few-objects}', 'workloads/write_fadvise_dontneed}']

Dead: b'2026-02-24T15:11:03.074 INFO:tasks.ceph.mgr.x.trial171.stderr:2026-02-24T15:11:03.070+0000 7f3f6e1f8640 -1 mgr handle_mgr_signal  *** Got signal Terminated ***'
2 jobs: ['68068', '68096']
suites intersection: ['clusters/crimson-fixed-1', 'crimson-rados/rbd/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}']
suites union: ['clusters/crimson-fixed-1', 'crimson-rados/rbd/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'tasks/rbd_api_tests_old_format}', 'tasks/rbd_api_tests}']

Failure: Command failed (workunit test cephtool/test_crimson_dump_metrics.sh) on trial185 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_crimson_dump_metrics.sh'
2 jobs: ['68053', '68083']
suites intersection: ['all/osd-dump-metrics', 'crimson-rados/singleton/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'rados}']
suites union: ['all/osd-dump-metrics', 'crimson-rados/singleton/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'objectstore/seastore/rbm$/{crimson_seastore_rbm}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'rados}']

Timeout 120 running ceph --cluster ceph osd dump --format=json
1 jobs: ['68114']
suites: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'thrashers/{pggrow}', 'thrashosds-health', 'workloads/write_fadvise_dontneed}']

Failure: Command failed (workunit test rbd/test_librbd_python.sh) on trial127 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh'
4 jobs: ['68107', '68060', '68088', '68064']
suites intersection: ['clusters/crimson-fixed-1', 'crimson-rados/rbd/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph']
suites union: ['clusters/crimson-fixed-1', 'crimson-rados/rbd/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/rbm$/{crimson_seastore_rbm_2q}', 'objectstore/seastore/rbm$/{crimson_seastore_rbm}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'tasks/rbd_python_api_tests_old_format}', 'tasks/rbd_python_api_tests}']

Failure: reached maximum tries (500) after waiting for 3000 seconds
2 jobs: ['68046', '68082']
suites intersection: ['0-size-min-size-overrides/3-size-2-min-size', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'thrashers/{default}', 'thrashosds-health', 'workloads/radosbench-high-concurrency}']
suites union: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados/thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'thrashers/{default}', 'thrashosds-health', 'workloads/radosbench-high-concurrency}']

Failure: "2026-02-24T15:08:44.300640+0000 mon.a (mon.0) 189 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log
1 jobs: ['68110']
suites: ['clusters/crimson-fixed-2', 'crimson-rados/basic/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'tasks/readwrite}']

Failure: Command failed (workunit test rados/test.sh) on trial106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c8fda9e9b7a6ecf86711fef9f31d8fff56203e8f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh --crimson'
1 jobs: ['68073']
suites: ['clusters/crimson-fixed-2', 'crimson-rados/basic/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented}', 'tasks/rados_api_tests}']