2021-09-03T00:19:43.766 INFO:root:teuthology version: 1.1.0-df69d6db
2021-09-03T00:19:43.767 INFO:teuthology.results:subset: '1/3'
2021-09-03T00:19:43.768 INFO:teuthology.results:seed: '831'
2021-09-03T00:34:52.157 INFO:root:teuthology version: 1.1.0-df69d6db
2021-09-03T00:34:52.163 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-09-03T02:07:58.621 INFO:teuthology.results:Tests finished! gathering results...
2021-09-03T02:07:59.660 INFO:scrape:Found 16 jobs
2021-09-03T02:08:00.283 INFO:scrape:Found 2 distinct failure reasons
2021-09-03T02:08:00.284 INFO:scrape:Failure: "2021-09-03T00:36:57.169326+0000 mon.a (mon.0) 301 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log
2021-09-03T02:08:00.284 INFO:scrape:2 jobs: ['6373066', '6373074']
2021-09-03T02:08:00.285 INFO:scrape:suites intersection: ['centos_latest', 'clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw/verify/{0-install', 's3tests-java', 's3tests}', 'tasks/{cls', 'validater/lockdep}']
2021-09-03T02:08:00.285 INFO:scrape:suites union: ['centos_latest', 'clusters/fixed-2', 'datacache/no_datacache', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{cls', 'validater/lockdep}']
2021-09-03T02:08:00.286 INFO:scrape:
2021-09-03T02:08:00.286 INFO:scrape:Failure: Command failed on smithi114 with status 1: 'sudo yum install -y kernel'
2021-09-03T02:08:00.286 INFO:scrape:1 jobs: ['6373069']
2021-09-03T02:08:00.287 INFO:scrape:suites: ['centos_latest', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{cls', 'validater/valgrind}']
2021-09-03T02:08:00.287 INFO:scrape:
2021-09-03T02:08:00.288 INFO:teuthology.results:Sending results to ivancich@redhat.com: Test Run: ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/
failed:  3
dead:    0
running: 0
waiting: 0
queued:  0
passed:  13


Fail
=================================================================
[6373066]  rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:27:00
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373066/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373066/

    "2021-09-03T00:36:57.169326+0000 mon.a (mon.0) 301 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6373069]  rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373069/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373069/
sentry: https://sentry.ceph.com/organizations/ceph/?query=029f7e6b7a3a4a5396483dd010e5cc20

    Command failed on smithi114 with status 1: 'sudo yum install -y kernel'

[6373074]  rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:18:37
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373074/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373074/
sentry: https://sentry.ceph.com/organizations/ceph/?query=398a75ebbecc403b8b63557597929445

    "2021-09-03T00:45:15.161879+0000 mon.a (mon.0) 115 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log



Pass
=================================================================
[6373064] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:25:39
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373064/

[6373065] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:28:37
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373065/

[6373067] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:12:00
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373067/

[6373068] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:27:24
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373068/

[6373070] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:26:55
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373070/

[6373071] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:16:38
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373071/

[6373072] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:27:36
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373072/

[6373073] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:20:58
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373073/

[6373075] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:22:46
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373075/

[6373076] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:26:31
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373076/

[6373077] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:25:48
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373077/

[6373078] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:28:58
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373078/

[6373079] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:11:22
info:   http://pulpito.ceph.com/ivancich-2021-09-03_00:17:55-rgw-wip-fix-bi-test-error-distro-basic-smithi/6373079/