Found 334 jobs
Found 5 distinct failure reasons
Failure: reached maximum tries (180) after waiting for 180 seconds
2 jobs: ['5238543', '5238769']
suites intersection: ['ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'distro$/{centos_7.6}', 'rados', 'rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'thrashosds-health', 'three-plus-one}']
suites union: ['1-install/jewel', '1-install/nautilus-v2only', 'backoff/normal', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'distro$/{centos_7.6}', 'msgr-failures/fastclose', 'msgr-failures/osd-delay', 'rados', 'rados/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'thrashers/default', 'thrashers/pggrow', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-snaps}', 'workloads/radosbench}']

Failure: wait_for_recovery: failed before timeout expired
1 jobs: ['5238348']
suites: ['1-install/mimic', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{rbd-cls', '5-workload/{radosbench', '6-finish-upgrade', '7-octopus', '8-workload/{rbd-python', 'bluestore-bitmap', 'rados/upgrade/mimic-x-singleton/{0-cluster/{openstack', 'rbd-import-export', 'rbd_api}', 'readwrite', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_latest}']

Failure: Command failed on smithi145 with status 1: 'sudo yum -y install ceph-mgr-diskprediction-local'
1 jobs: ['5238682']
suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'validater/valgrind}']

Failure: "2020-07-20T06:39:24.281059+0000 mon.a (mon.0) 64 : cluster [ERR] Health check failed: 4 mgr modules have failed (MGR_MODULE_ERROR)" in cluster log
1 jobs: ['5238453']
suites: ['msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-stupid', 'rados', 'rados/singleton/{all/rebuild-mondb', 'supported-random-distro$/{rhel_latest}}']

Failure: Command failed on smithi141 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:bdc0fb0c5eb43ffa20f86c88093d654732d1e1ac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f4345526-ca68-11ea-a06f-001a4aab830c -- ceph orch host add smithi141'
1 jobs: ['5238712']
suites: ['1-install/jewel-v1only', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/off', 'distro$/{centos_7.6}', 'msgr-failures/osd-delay', 'rados', 'rados/thrash-old-clients/{0-size-min-size-overrides/3-size-2-min-size', 'thrashers/none', 'thrashosds-health', 'three-plus-one}', 'workloads/test_rbd_api}']