2023-12-03T10:46:30.207 INFO:root:teuthology version: 0.0.1.dev214+gf2de108
2023-12-03T10:46:30.208 INFO:teuthology.results:seed: '9570'
2023-12-03T10:52:19.988 INFO:root:teuthology version: 0.0.1.dev214+gf2de108
2023-12-03T10:52:19.992 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-12-03T22:53:13.288 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds
2023-12-03T22:53:16.040 INFO:scrape:Found 34 jobs
2023-12-03T22:53:18.179 INFO:scrape:Found 11 distinct failure reasons
2023-12-03T22:53:18.179 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/test_lock_fence.sh
2023-12-03T22:53:18.180 INFO:scrape:1 jobs: ['7476089']
2023-12-03T22:53:18.180 INFO:scrape:suites: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_lock_and_fence}']
2023-12-03T22:53:18.180 INFO:scrape:
2023-12-03T22:53:18.180 INFO:scrape:Dead: b'2023-12-03T11:16:35.592 INFO:tasks.ceph.mon.a.smithi141.stderr:2023-12-03T11:16:35.591+0000 7f60d51fc640 -1 mon.a@0(leader) e1 *** Got Signal Terminated ***'
2023-12-03T22:53:18.180 INFO:scrape:5 jobs: ['7476071', '7476093', '7476076', '7476097', '7476082']
2023-12-03T22:53:18.180 INFO:scrape:suites intersection: ['crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore']
2023-12-03T22:53:18.181 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{fixed-2}', 'crimson-rados/perf/{clusters/{fixed-2}', 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'settings/optimized', 'thrashers/default', 'thrashosds-health', 'workloads/fio_4K_rand_read}', 'workloads/fio_4M_rand_write}', 'workloads/pool-snaps-few-objects}', 'workloads/snaps-few-objects-balanced}', 'workloads/snaps-few-objects-localized}']
2023-12-03T22:53:18.181 INFO:scrape:
2023-12-03T22:53:18.181 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/test_librbd_python.sh -m \
2023-12-03T22:53:18.181 INFO:scrape:2 jobs: ['7476095', '7476100']
2023-12-03T22:53:18.181 INFO:scrape:suites intersection: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph']
2023-12-03T22:53:18.181 INFO:scrape:suites union: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_python_api_tests_old_format}', 'tasks/rbd_python_api_tests}']
2023-12-03T22:53:18.181 INFO:scrape:
2023-12-03T22:53:18.181 INFO:scrape:Dead: b'2023-12-03T22:52:43.764 INFO:teuthology.orchestra.run.smithi100.stderr:2023-12-03T22:52:43.762+0000 7fe277aad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson'
2023-12-03T22:53:18.181 INFO:scrape:2 jobs: ['7476099', '7476087']
2023-12-03T22:53:18.182 INFO:scrape:suites intersection: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{fixed-2}', 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'thrashers/default', 'thrashosds-health']
2023-12-03T22:53:18.182 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{fixed-2}', 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'thrashers/default', 'thrashosds-health', 'workloads/small-objects-localized}', 'workloads/snaps-few-objects}']
2023-12-03T22:53:18.182 INFO:scrape:
2023-12-03T22:53:18.182 INFO:scrape:Failure: Failed to reconnect to smithi131
2023-12-03T22:53:18.182 INFO:scrape:1 jobs: ['7476085']
2023-12-03T22:53:18.182 INFO:scrape:suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{fixed-2}', 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'thrashers/default', 'thrashosds-health', 'workloads/small-objects-balanced}']
2023-12-03T22:53:18.182 INFO:scrape:
2023-12-03T22:53:18.182 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/crimson/test_crimson_librbd.sh
2023-12-03T22:53:18.182 INFO:scrape:2 jobs: ['7476072', '7476078']
2023-12-03T22:53:18.182 INFO:scrape:suites intersection: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph']
2023-12-03T22:53:18.183 INFO:scrape:suites union: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_api_tests_old_format}', 'tasks/rbd_api_tests}']
2023-12-03T22:53:18.183 INFO:scrape:
2023-12-03T22:53:18.183 INFO:scrape:Timeout 1h running clone.client.0/qa/workunits/rados/test_python.sh -m \
2023-12-03T22:53:18.183 INFO:scrape:1 jobs: ['7476083']
2023-12-03T22:53:18.183 INFO:scrape:suites: ['crimson-rados/basic/{clusters/fixed-2', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rados_python}']
2023-12-03T22:53:18.183 INFO:scrape:
2023-12-03T22:53:18.183 INFO:scrape:Failure: Command failed on smithi131 with status 1: 'sudo yum install -y kernel'
2023-12-03T22:53:18.183 INFO:scrape:1 jobs: ['7476086']
2023-12-03T22:53:18.183 INFO:scrape:suites: ['crimson-rados/perf/{clusters/{fixed-2}', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'settings/optimized', 'workloads/radosbench_4K_rand_read}']
2023-12-03T22:53:18.183 INFO:scrape:
2023-12-03T22:53:18.183 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/cls/test_cls_rbd.sh
2023-12-03T22:53:18.183 INFO:scrape:1 jobs: ['7476084']
2023-12-03T22:53:18.184 INFO:scrape:suites: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_cls_tests}']
2023-12-03T22:53:18.184 INFO:scrape:
2023-12-03T22:53:18.184 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh
2023-12-03T22:53:18.184 INFO:scrape:1 jobs: ['7476070']
2023-12-03T22:53:18.184 INFO:scrape:suites: ['crimson-rados/basic/{clusters/fixed-2', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rados_api_tests}']
2023-12-03T22:53:18.184 INFO:scrape:
2023-12-03T22:53:18.184 INFO:scrape:Dead: b'2023-12-03T22:53:11.910 DEBUG:teuthology.exit:Finished running handlers'
2023-12-03T22:53:18.184 INFO:scrape:1 jobs: ['7476091']
2023-12-03T22:53:18.184 INFO:scrape:suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{fixed-2}', 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'thrashers/default', 'thrashosds-health', 'workloads/small-objects}']
2023-12-03T22:53:18.184 INFO:scrape: