Found 22 jobs Found 3 distinct failure reasons Failure: reached maximum tries (500) after waiting for 3000 seconds 1 jobs: ['62664'] suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-active-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', 'backoff/peering_and_degraded', 'base', 'ceph', 'clusters/{fixed-4}', 'crc-failures/default', 'd-balancer/on', 'mem$/{normal-2}', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/{bluestore/{alloc$/{hybrid}', 'onode-segment$/{512K}', 'rados', 'rados:thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'thrashers/careful_host', 'thrashosds-health', 'v1}}}}', 'workloads/radosbench-high-concurrency}', 'write$/{v1/{compr$/{yes$/{lz4}}'] Failure: The WatchDog barked due to Daemon ceph.osd.12 has failed 1 jobs: ['62682'] suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-async-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-1}', 'backoff/normal', 'base', 'ceph', 'clusters/{fixed-4}', 'crc-failures/default', 'd-balancer/upmap-read', 'mem$/{normal-1}', 'mon_election/connectivity', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v1only', 'objectstore/{bluestore/{alloc$/{btree}', 'onode-segment$/{none}', 'rados', 'rados:thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'thrashers/mapgap_host', 'thrashosds-health', 'v1}}}}', 'workloads/pool-snaps-few-objects}', 'write$/{v1/{compr$/{no$/{no}}'] Failure: Command failed (workunit test rados/test.sh) on trial187 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7eb1e7b478bae35f04b0b595f6e5c1e0a5b1f998 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 1 jobs: ['62683'] suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-async-partial-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-5}', 'backoff/peering', 'base', 'ceph', 'clusters/{fixed-4}', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'mem$/{low}', 'mon_election/classic', 'msgr-failures/fastclose', 'msgr/async-v2only', 'objectstore/{bluestore/{alloc$/{stupid}', 'onode-segment$/{512K-onoff}', 'rados', 'rados:thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{centos_latest}', 'thrashers/morepggrow', 'thrashosds-health', 'v2}}}}', 'workloads/rados_api_tests}', 'write$/{v2/{compr$/{yes$/{zstd}}']