Found 34 jobs Missing teuthology log /home/teuthworker/archive/yingxin-2024-03-15_08:48:17-crimson-rados-ci-yingxin-crimson-no-cluster-mcore-msgr-distro-default-smithi/7601845/teuthology.log Found 5 distinct failure reasons Failure: "2024-03-15T09:57:57.467296+0000 mon.a (mon.0) 659 : cluster [ERR] Health check failed: 9 scrub errors (OSD_SCRUB_ERRORS)" in cluster log 6 jobs: ['7601871', '7601860', '7601855', '7601865', '7601849', '7601866'] suites intersection: ['crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph'] suites union: ['crimson-rados/basic/{clusters/fixed-2', 'crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_api_tests_old_format}', 'tasks/rbd_cls_tests}', 'tasks/rbd_lock_and_fence}', 'tasks/rbd_python_api_tests_old_format}', 'tasks/rbd_python_api_tests}', 'tasks/readwrite}'] Timeout 120 running ceph --cluster ceph pg dump --format=json 1 jobs: ['7601854'] suites: ['crimson-rados/basic/{clusters/fixed-2', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rados_python}'] Timeout 3h running clone.client.0/qa/workunits/rados/test.sh --crimson 1 jobs: ['7601841'] suites: ['crimson-rados/basic/{clusters/fixed-2', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rados_api_tests}'] Dead: None 1 jobs: ['7601845'] Dead: b'2024-03-15T21:18:36.116 DEBUG:teuthology.exit:Finished running handlers' 1 jobs: ['7601859'] suites: ['crimson-rados/perf/{clusters/{fixed-2}', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'settings/optimized', 'workloads/radosbench_4K_seq_read}']