Found 68 jobs
Found 5 distinct failure reasons
Failure: 
27 jobs: ['7384233', '7384277', '7384221', '7384274', '7384246', '7384264', '7384282', '7384252', '7384251', '7384222', '7384265', '7384226', '7384236', '7384283', '7384253', '7384278', '7384254', '7384245', '7384227', '7384271', '7384259', '7384235', '7384250', '7384239', '7384229', '7384257', '7384247']
suites intersection: ['crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides']
suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{fixed-2}', 'crimson-rados/basic/{clusters/fixed-2', 'crimson-rados/perf/{clusters/{fixed-2}', 'crimson-rados/rbd/{clusters/fixed-1', 'crimson-rados/singleton/{all/osd-backfill', 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'rados}', 'settings/optimized', 'tasks/rados_api_tests}', 'tasks/rados_python}', 'tasks/rbd_api_tests}', 'tasks/rbd_cls_tests}', 'tasks/rbd_lock_and_fence}', 'tasks/rbd_python_api_tests_old_format}', 'tasks/readwrite}', 'thrashers/default', 'thrashosds-health', 'workloads/admin_socket_objecter_requests}', 'workloads/fio_4K_rand_read}', 'workloads/fio_4K_rand_rw}', 'workloads/fio_4M_rand_read}', 'workloads/fio_4M_rand_rw}', 'workloads/pool-snaps-few-objects}', 'workloads/radosbench-high-concurrency}', 'workloads/radosbench_4M_rand_read}', 'workloads/radosbench_omap_write}', 'workloads/radosbench}', 'workloads/sample_fio}', 'workloads/sample_radosbench}', 'workloads/small-objects-balanced}', 'workloads/small-objects-localized}', 'workloads/small-objects}', 'workloads/snaps-few-objects-balanced}', 'workloads/snaps-few-objects-localized}', 'workloads/snaps-few-objects}', 'workloads/write_fadvise_dontneed}']

Timeout 3h running clone.client.0/qa/workunits/rbd/test_librbd_python.sh -m \
1 jobs: ['7384275']
suites: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_python_api_tests}']

Failure: "2023-08-30T13:51:28.788604+0000 mon.a (mon.0) 154 : cluster [WRN] Health check failed: Degraded data redundancy: 81/9356 objects degraded (0.866%), 5 pgs degraded (PG_DEGRADED)" in cluster log
4 jobs: ['7384232', '7384242', '7384268', '7384272']
suites intersection: ['crimson-rados/perf/{clusters/{fixed-2}', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'settings/optimized']
suites union: ['crimson-rados/perf/{clusters/{fixed-2}', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'settings/optimized', 'workloads/radosbench_4K_rand_read}', 'workloads/radosbench_4K_seq_read}', 'workloads/radosbench_4M_seq_read}', 'workloads/radosbench_4M_write}']

Timeout 3h running clone.client.0/qa/workunits/rbd/crimson/test_crimson_librbd.sh
1 jobs: ['7384224']
suites: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_api_tests_old_format}']

Failure: Command failed (workunit test rados/test.sh) on smithi096 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b8902c0e861a914e1c434df45f3f7605da267573 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'
1 jobs: ['7384216']
suites: ['crimson-rados/basic/{clusters/fixed-2', 'crimson-supported-all-distro/centos_8', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rados_api_tests}']