Found 24 jobs Missing teuthology log /home/teuthworker/archive/rfriedma-2024-03-08_18:16:19-rados:thrash-wip-rf-compactref-distro-default-smithi/7588078/teuthology.log Found 3 distinct failure reasons Failure: reached maximum tries (31) after waiting for 300 seconds 8 jobs: ['7588083', '7588087', '7588091', '7588072', '7588090', '7588080', '7588074', '7588073'] suites intersection: ['1-pg-log-overrides/normal_pg_log', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/on', 'mon_election/classic', 'msgr-failures/osd-dispatch-delay', 'openstack}', 'rados', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'thrashosds-health'] suites union: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-active-recovery}', '2-recovery-overrides/{more-async-recovery}', '2-recovery-overrides/{more-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-1}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', '3-scrub-overrides/{max-simultaneous-scrubs-5}', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/on', 'mon_election/classic', 'msgr-failures/osd-dispatch-delay', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'thrashers/careful', 'thrashers/mapgap', 'thrashers/none', 'thrashosds-health', 'workloads/cache-agent-big}', 'workloads/pool-snaps-few-objects}', 'workloads/radosbench}', 'workloads/redirect_set_object}', 'workloads/set-chunks-read}', 'workloads/small-objects}', 'workloads/snaps-few-objects}', 'workloads/write_fadvise_dontneed}'] Dead: None 1 jobs: ['7588078'] Timeout 3h running clone.client.0/qa/workunits/rados/test.sh 1 jobs: ['7588077'] suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-5}', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/on', 'mon_election/classic', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v2only', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/mapgap', 'thrashosds-health', 'workloads/rados_api_tests}']