2024-04-11T13:05:53.278 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-04-11T13:05:53.278 INFO:teuthology.results:seed: '4301' 2024-04-11T13:36:45.771 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-04-11T13:36:45.790 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-04-11T16:56:01.168 INFO:teuthology.results:Tests finished! gathering results... 2024-04-11T16:56:03.495 INFO:scrape:Found 34 jobs 2024-04-11T16:56:04.808 INFO:scrape:Found 5 distinct failure reasons 2024-04-11T16:56:04.809 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/test_librbd_python.sh -m \ 2024-04-11T16:56:04.809 INFO:scrape:2 jobs: ['7652285', '7652280'] 2024-04-11T16:56:04.809 INFO:scrape:suites intersection: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph'] 2024-04-11T16:56:04.809 INFO:scrape:suites union: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_python_api_tests_old_format}', 'tasks/rbd_python_api_tests}'] 2024-04-11T16:56:04.809 INFO:scrape: 2024-04-11T16:56:04.809 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/crimson/test_crimson_librbd.sh 2024-04-11T16:56:04.809 INFO:scrape:2 jobs: ['7652257', '7652263'] 2024-04-11T16:56:04.809 INFO:scrape:suites intersection: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph'] 2024-04-11T16:56:04.809 INFO:scrape:suites union: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_api_tests_old_format}', 'tasks/rbd_api_tests}'] 2024-04-11T16:56:04.809 INFO:scrape: 2024-04-11T16:56:04.810 INFO:scrape:Failure: "2024-04-11T13:53:13.376576+0000 mon.a (mon.0) 165 : cluster [ERR] Health check failed: 9 scrub errors (OSD_SCRUB_ERRORS)" in cluster log 2024-04-11T16:56:04.810 INFO:scrape:3 jobs: ['7652279', '7652269', '7652274'] 2024-04-11T16:56:04.810 INFO:scrape:suites intersection: ['crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph'] 2024-04-11T16:56:04.810 INFO:scrape:suites union: ['crimson-rados/basic/{clusters/fixed-2', 'crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_cls_tests}', 'tasks/rbd_lock_and_fence}', 'tasks/readwrite}'] 2024-04-11T16:56:04.810 INFO:scrape: 2024-04-11T16:56:04.810 INFO:scrape:Timeout 120 running ceph --cluster ceph pg dump --format=json 2024-04-11T16:56:04.810 INFO:scrape:1 jobs: ['7652281'] 2024-04-11T16:56:04.810 INFO:scrape:suites: ['crimson-rados/perf/{clusters/{fixed-2}', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'settings/optimized', 'workloads/radosbench_4M_write}'] 2024-04-11T16:56:04.810 INFO:scrape: 2024-04-11T16:56:04.810 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh --crimson 2024-04-11T16:56:04.811 INFO:scrape:1 jobs: ['7652255'] 2024-04-11T16:56:04.811 INFO:scrape:suites: ['crimson-rados/basic/{clusters/fixed-2', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rados_api_tests}'] 2024-04-11T16:56:04.811 INFO:scrape: