Found 48 jobs Found 4 distinct failure reasons Failure: Command failed (s3 tests against rgw) on smithi005 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain'" 40 jobs suites intersection: ['cls', 'clusters/fixed-2', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 's3tests-java', 's3tests}', 'tasks/{0-install'] suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}'] Failure: "2020-07-08T19:00:00.000116+0000 mon.b (mon.0) 113 : cluster [WRN] overall HEALTH_WARN Reduced data availability: 1 pg inactive, 1 pg peering" in cluster log 4 jobs: ['5209660', '5209702', '5209684', '5209683'] suites intersection: ['cls', 'clusters/fixed-2', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 's3tests-java', 's3tests}', 'sharding$/{single}', 'tasks/{0-install'] suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}'] Failure: Scrubbing terminated -- not all pgs were active and clean. 1 jobs: ['5209688'] suites: ['cls', 'clusters/fixed-2', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{default}', 'striping$/{stripe-equals-chunk}', 'tasks/{0-install', 'validater/lockdep}'] Failure: "2020-07-08T19:07:27.219457+0000 osd.2 (osd.2) 3 : cluster [WRN] slow request osd_op(client.4156.0:1 2.1 2.3e6a8311 (undecoded) ondisk+read+known_if_redirected e18) initiated 2020-07-08T19:06:56.541574+0000 currently delayed" in cluster log 3 jobs: ['5209696', '5209678', '5209691'] suites intersection: ['cls', 'clusters/fixed-2', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 's3tests-java', 's3tests}', 'tasks/{0-install'] suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}']