Found 16 jobs Found 4 distinct failure reasons Failure: 7 jobs: ['6379633', '6379622', '6379635', '6379624', '6379628', '6379626', '6379631'] suites intersection: ['cls', 'clusters/fixed-2', 'frontend/civetweb', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 's3tests-java', 's3tests}', 'tasks/{0-install'] suites union: ['cls', 'clusters/fixed-2', 'frontend/civetweb', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}'] Failure: "2021-09-08T04:19:23.194012+0000 mon.a (mon.0) 117 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log 1 jobs: ['6379632'] suites: ['cls', 'clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/valgrind}'] Failure: Command failed (workunit test cls/test_cls_rgw.sh) on smithi122 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e216c1ee110397eb48ef026c794021917db737f2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh' 4 jobs: ['6379630', '6379625', '6379623', '6379627'] suites intersection: ['cls', 'clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 's3tests-java', 's3tests}', 'tasks/{0-install'] suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}'] Failure: reached maximum tries (90) after waiting for 540 seconds 1 jobs: ['6379629'] suites: ['cls', 'clusters/fixed-2', 'frontend/civetweb', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{0-install', 'validater/lockdep}']