2024-04-16T07:26:28.609 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-04-16T07:26:28.610 INFO:teuthology.results:seed: '6444' 2024-04-16T07:27:29.570 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-04-16T07:27:29.575 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-04-16T19:28:23.804 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2024-04-16T19:28:27.165 INFO:scrape:Found 34 jobs 2024-04-16T19:28:28.535 INFO:scrape:Found 7 distinct failure reasons 2024-04-16T19:28:28.535 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/crimson/test_crimson_librbd.sh 2024-04-16T19:28:28.535 INFO:scrape:2 jobs: ['7657987', '7657981'] 2024-04-16T19:28:28.535 INFO:scrape:suites intersection: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph'] 2024-04-16T19:28:28.536 INFO:scrape:suites union: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_api_tests_old_format}', 'tasks/rbd_api_tests}'] 2024-04-16T19:28:28.536 INFO:scrape: 2024-04-16T19:28:28.536 INFO:scrape:Dead: b'2024-04-16T19:27:45.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-04-16T19:27:45.742+0000 7f5db4c6d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson' 2024-04-16T19:28:28.536 INFO:scrape:7 jobs: ['7658002', '7658000', '7657985', '7657994', '7658006', '7658008', '7657996'] 2024-04-16T19:28:28.536 INFO:scrape:suites intersection: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{fixed-2}', 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'thrashers/default', 'thrashosds-health'] 2024-04-16T19:28:28.536 INFO:scrape:suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{fixed-2}', 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'thrashers/default', 'thrashosds-health', 'workloads/pool-snaps-few-objects}', 'workloads/small-objects-balanced}', 'workloads/small-objects-localized}', 'workloads/small-objects}', 'workloads/snaps-few-objects-balanced}', 'workloads/snaps-few-objects-localized}', 'workloads/snaps-few-objects}'] 2024-04-16T19:28:28.536 INFO:scrape: 2024-04-16T19:28:28.536 INFO:scrape:Failure: Command failed on smithi194 with status 1: 'sudo yum install -y kernel' 2024-04-16T19:28:28.536 INFO:scrape:1 jobs: ['7657993'] 2024-04-16T19:28:28.537 INFO:scrape:suites: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_cls_tests}'] 2024-04-16T19:28:28.537 INFO:scrape: 2024-04-16T19:28:28.537 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh --crimson 2024-04-16T19:28:28.537 INFO:scrape:1 jobs: ['7657979'] 2024-04-16T19:28:28.537 INFO:scrape:suites: ['crimson-rados/basic/{clusters/fixed-2', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rados_api_tests}'] 2024-04-16T19:28:28.537 INFO:scrape: 2024-04-16T19:28:28.537 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/test_librbd_python.sh -m \ 2024-04-16T19:28:28.537 INFO:scrape:2 jobs: ['7658009', '7658004'] 2024-04-16T19:28:28.537 INFO:scrape:suites intersection: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph'] 2024-04-16T19:28:28.537 INFO:scrape:suites union: ['crimson-rados/rbd/{clusters/fixed-1', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rbd_python_api_tests_old_format}', 'tasks/rbd_python_api_tests}'] 2024-04-16T19:28:28.537 INFO:scrape: 2024-04-16T19:28:28.537 INFO:scrape:Failure: Failed to reconnect to smithi194 2024-04-16T19:28:28.537 INFO:scrape:1 jobs: ['7657992'] 2024-04-16T19:28:28.538 INFO:scrape:suites: ['crimson-rados/basic/{clusters/fixed-2', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rados_python}'] 2024-04-16T19:28:28.538 INFO:scrape: 2024-04-16T19:28:28.538 INFO:scrape:Timeout 120 running ceph --cluster ceph pg dump --format=json 2024-04-16T19:28:28.538 INFO:scrape:1 jobs: ['7658005'] 2024-04-16T19:28:28.538 INFO:scrape:suites: ['crimson-rados/perf/{clusters/{fixed-2}', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'settings/optimized', 'workloads/radosbench_4M_write}'] 2024-04-16T19:28:28.538 INFO:scrape: