Found 29 jobs Found 4 distinct failure reasons Failure: timed out waiting for mon to be updated with osd.0: 47244641477 < 47244641484 1 jobs: ['46153'] suites: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados:thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'thrashers/{pggrow}', 'thrashosds-health', 'workloads/pool-snaps-few-objects}'] Failure: reached maximum tries (500) after waiting for 3000 seconds 3 jobs: ['46154', '46155', '46149'] suites intersection: ['0-size-min-size-overrides/3-size-2-min-size', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados:thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'thrashers/{pggrow}', 'thrashosds-health'] suites union: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados:thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/crimson_bluestore', 'objectstore/seastore/rbm$/{crimson_seastore_rbm_2q}', 'objectstore/seastore/rbm$/{crimson_seastore_rbm}', 'thrashers/{pggrow}', 'thrashosds-health', 'workloads/radosbench-high-concurrency}', 'workloads/radosbench}'] Timeout 120 running ceph --cluster ceph osd dump --format=json 2 jobs: ['46144', '46136'] suites intersection: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados:thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'thrashers/{pggrow}', 'thrashosds-health'] suites union: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados:thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/rbm$/{crimson_seastore_rbm_2q}', 'objectstore/seastore/rbm$/{crimson_seastore_rbm}', 'thrashers/{pggrow}', 'thrashosds-health', 'workloads/radosbench-high-concurrency}', 'workloads/radosbench}'] Failure: Command failed on trial109 with status 6: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats' 1 jobs: ['46142'] suites: ['0-size-min-size-overrides/3-size-2-min-size', '1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{crimson-fixed-2}', 'crimson-rados:thrash/{0-crimson_install', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/seastore/segmented$/{crimson_seastore_segmented_2q}', 'thrashers/{pggrow}', 'thrashosds-health', 'workloads/snaps-few-objects}']