2020-07-15T09:26:45.211 INFO:root:teuthology version: 1.0.0-da68153 2020-07-15T09:26:45.213 INFO:teuthology.results:subset: '520/1677' 2020-07-15T09:26:45.214 INFO:teuthology.results:seed: '5162' 2020-07-15T14:18:01.789 INFO:root:teuthology version: 1.0.0-da68153 2020-07-15T14:18:01.813 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-07-15T16:38:06.231 INFO:teuthology.results:Tests finished! gathering results... 2020-07-15T16:38:27.012 INFO:scrape:Found 243 jobs 2020-07-15T16:38:28.957 INFO:scrape:Found 3 distinct failure reasons 2020-07-15T16:38:28.958 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rados/test.sh 2020-07-15T16:38:28.958 INFO:scrape:1 jobs: ['5228867'] 2020-07-15T16:38:28.959 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log.yaml', '2-recovery-overrides/{more-async-partial-recovery.yaml}', 'backoff/peering_and_degraded.yaml', 'ceph.yaml', 'clusters/{fixed-2.yaml', 'd-balancer/crush-compat.yaml', 'msgr-failures/osd-delay.yaml', 'msgr/random.yaml', 'objectstore/bluestore-comp-zlib.yaml', 'openstack.yaml}', 'rados.yaml', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml', 'supported-random-distro$/{ubuntu_16.04.yaml}', 'thrashers/default.yaml', 'thrashosds-health.yaml', 'workloads/rados_api_tests.yaml}'] 2020-07-15T16:38:28.959 INFO:scrape: 2020-07-15T16:38:28.960 INFO:scrape:Failure: "2020-07-15 13:28:17.508125 mon.b (mon.1) 9 : cluster [WRN] Health check failed: 1/3 mons down, quorum b,c (MON_DOWN)" in cluster log 2020-07-15T16:38:28.960 INFO:scrape:1 jobs: ['5228874'] 2020-07-15T16:38:28.961 INFO:scrape:suites: ['1-pg-log-overrides/normal_pg_log.yaml', '2-recovery-overrides/{more-async-recovery.yaml}', 'backoff/normal.yaml', 'ceph.yaml', 'clusters/{fixed-2.yaml', 'd-balancer/off.yaml', 'msgr-failures/fastclose.yaml', 'msgr/async-v1only.yaml', 'objectstore/bluestore-comp-zlib.yaml', 'openstack.yaml}', 'rados.yaml', 'rados:thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml', 'supported-random-distro$/{ubuntu_latest.yaml}', 'thrashers/mapgap.yaml', 'thrashosds-health.yaml', 'workloads/small-objects.yaml}'] 2020-07-15T16:38:28.961 INFO:scrape: 2020-07-15T16:38:28.962 INFO:scrape:Failure: reached maximum tries (500) after waiting for 3000 seconds 2020-07-15T16:38:28.962 INFO:scrape:1 jobs: ['5228923'] 2020-07-15T16:38:28.963 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log.yaml', '2-recovery-overrides/{more-active-recovery.yaml}', 'backoff/peering.yaml', 'ceph.yaml', 'clusters/{fixed-2.yaml', 'd-balancer/upmap.yaml', 'msgr-failures/few.yaml', 'msgr/simple.yaml', 'objectstore/bluestore-comp-zlib.yaml', 'openstack.yaml}', 'rados.yaml', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size.yaml', 'supported-random-distro$/{ubuntu_16.04.yaml}', 'thrashers/mapgap.yaml', 'thrashosds-health.yaml', 'workloads/radosbench.yaml}'] 2020-07-15T16:38:28.963 INFO:scrape: