2020-07-19T13:18:19.777 INFO:root:teuthology version: 1.0.0-0084e27
2020-07-19T13:18:19.779 INFO:teuthology.results:seed: '3589'
2020-07-19T13:46:07.031 INFO:root:teuthology version: 1.0.0-0084e27
2020-07-19T13:46:07.060 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-07-19T14:53:08.846 INFO:teuthology.results:Tests finished! gathering results...
2020-07-19T14:53:13.271 INFO:scrape:Found 48 jobs
2020-07-19T14:53:27.368 INFO:scrape:Found 4 distinct failure reasons
2020-07-19T14:53:27.369 INFO:scrape:Failure: Command failed (s3 tests against rgw) on smithi124 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!s3select,!fails_with_subdomain'"
2020-07-19T14:53:27.370 INFO:scrape:30 jobs
2020-07-19T14:53:27.371 INFO:scrape:suites intersection: ['cls', 'clusters/fixed-2', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 's3tests-java', 's3tests}', 'tasks/{0-install']
2020-07-19T14:53:27.371 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}']
2020-07-19T14:53:27.372 INFO:scrape:
2020-07-19T14:53:27.372 INFO:scrape:Failure: "2020-07-19T14:00:00.000170+0000 mon.a (mon.0) 149 : cluster [WRN] overall HEALTH_WARN Reduced data availability: 1 pg inactive, 1 pg peering" in cluster log
2020-07-19T14:53:27.373 INFO:scrape:4 jobs: ['5239926', '5239899', '5239896', '5239913']
2020-07-19T14:53:27.373 INFO:scrape:suites intersection: ['cls', 'clusters/fixed-2', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 's3tests-java', 's3tests}', 'sharding$/{default}', 'tasks/{0-install']
2020-07-19T14:53:27.374 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{default}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}']
2020-07-19T14:53:27.374 INFO:scrape:
2020-07-19T14:53:27.374 INFO:scrape:Failure: "2020-07-19T13:47:15.121859+0000 osd.7 (osd.7) 3 : cluster [WRN] slow request osd_op(client.4115.0:2 2.1 2.3e6a8311 (undecoded) ondisk+read+known_if_redirected e18) initiated 2020-07-19T13:46:44.916853+0000 currently delayed" in cluster log
2020-07-19T14:53:27.375 INFO:scrape:2 jobs: ['5239900', '5239910']
2020-07-19T14:53:27.375 INFO:scrape:suites intersection: ['cls', 'clusters/fixed-2', 'msgr-failures/few', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 's3tests-java', 's3tests}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/valgrind}']
2020-07-19T14:53:27.376 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/valgrind}']
2020-07-19T14:53:27.376 INFO:scrape:
2020-07-19T14:53:27.377 INFO:scrape:Failure: Scrubbing terminated -- not all pgs were active and clean.
2020-07-19T14:53:27.377 INFO:scrape:1 jobs: ['5239904']
2020-07-19T14:53:27.377 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw:verify/{centos_latest', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/valgrind}']
2020-07-19T14:53:27.378 INFO:scrape:
2020-07-19T14:53:27.381 INFO:teuthology.results:Sending results to gsalomon@redhat.com: Test Run: gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/
failed:  37
dead:    0
running: 0
waiting: 0
queued:  0
passed:  11


Fail
=================================================================
[5239884]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:29:36
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239884/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239884/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=7c06c4c31e6b4a1daf79ab0301ec428b

    Command failed (s3 tests against rgw) on smithi133 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239886]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:29:38
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239886/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239886/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=a8a236b540fb45148229e4a728883e63

    Command failed (s3 tests against rgw) on smithi181 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239887]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:27:34
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239887/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239887/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=f9d76be9bd0741158da504d9dd3a9e86

    Command failed (s3 tests against rgw) on smithi017 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239888]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:29:15
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239888/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239888/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=c01bac57f9fd49ccaaa827c54d5ba6d6

    Command failed (s3 tests against rgw) on smithi050 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239890]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:26:38
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239890/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239890/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=4e63c73bf5894bde9e187a9e5b77d01b

    Command failed (s3 tests against rgw) on smithi049 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239891]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:29:25
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239891/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239891/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=9cc2f641cac349328d8f9127960eb190

    Command failed (s3 tests against rgw) on smithi120 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239892]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:27:56
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239892/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239892/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=b5672c232fad4bc0b0738cabe70ba0a3

    Command failed (s3 tests against rgw) on smithi129 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239895]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:27:02
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239895/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239895/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=8f7065502aad440581408a213697169f

    Command failed (s3 tests against rgw) on smithi173 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239896]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:41:43
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239896/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239896/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=67c20695bf3643d091acfa604c37acfe

    "2020-07-19T14:00:00.000185+0000 mon.b (mon.0) 113 : cluster [WRN] overall
    HEALTH_WARN Reduced data availability: 1 pg inactive, 1 pg peering" in
    cluster log

[5239897]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:26:37
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239897/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239897/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=fd4b6a49faf945c6b6bf134f12f06f6c

    Command failed (s3 tests against rgw) on smithi031 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239899]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:40:05
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239899/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239899/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=bf81c9b2bdc042079447e12c8386b55e

    "2020-07-19T14:00:00.000159+0000 mon.a (mon.0) 310 : cluster [WRN] overall
    HEALTH_WARN Reduced data availability: 2 pgs inactive, 2 pgs peering" in
    cluster log

[5239900]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:41:43
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239900/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239900/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=f78620ba88404f199290378d5685c6e9

    "2020-07-19T13:47:15.121859+0000 osd.7 (osd.7) 3 : cluster [WRN] slow
    request osd_op(client.4115.0:2 2.1 2.3e6a8311 (undecoded)
    ondisk+read+known_if_redirected e18) initiated
    2020-07-19T13:46:44.916853+0000 currently delayed" in cluster log

[5239901]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:27:03
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239901/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239901/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=b56b87adc81246f2a4facb3ccf61e516

    Command failed (s3 tests against rgw) on smithi125 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239902]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:29:47
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239902/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239902/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=d97f5d1e871d4e8da47839d252bcc485

    Command failed (s3 tests against rgw) on smithi002 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239903]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:26:05
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239903/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239903/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=c2607fa9069d49d2b48dbe22bbfa909a

    Command failed (s3 tests against rgw) on smithi138 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239904]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:42:01
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239904/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239904/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=542132fd545d42d79d20c8ad12159f41

    Scrubbing terminated -- not all pgs were active and clean.

[5239905]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:27:10
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239905/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239905/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=8dc0bca7b5d74c57b266ee51b3673697

    Command failed (s3 tests against rgw) on smithi133 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239906]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:29:31
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239906/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239906/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=240eb739cd1a4bffa39198116b264ef8

    Command failed (s3 tests against rgw) on smithi117 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239907]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:28:21
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239907/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239907/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=0a3648582bd340f88e074e7d826c94c2

    Command failed (s3 tests against rgw) on smithi082 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239908]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:28:56
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239908/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239908/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=9e73c80189164c42a84f14164f9184d8

    Command failed (s3 tests against rgw) on smithi140 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239909]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:27:52
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239909/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239909/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=c010647bae434e6e96e93f17b000d142

    Command failed (s3 tests against rgw) on smithi072 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239910]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:41:55
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239910/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239910/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=c04a8360ab464b33ac59372554d2f6ea

    "2020-07-19T13:52:08.848138+0000 osd.7 (osd.7) 3 : cluster [WRN] slow
    request osd_op(client.4152.0:2 2.1 2.3e6a8311 (undecoded)
    ondisk+read+known_if_redirected e18) initiated
    2020-07-19T13:51:38.423827+0000 currently delayed" in cluster log

[5239911]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:29:15
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239911/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239911/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=08a362674fa747388e21b66acc83f79d

    Command failed (s3 tests against rgw) on smithi124 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239913]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:40:37
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239913/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239913/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=c1fd058a4dc444b08b3a3b96ca035327

    "2020-07-19T14:00:00.000078+0000 mon.a (mon.0) 187 : cluster [WRN] overall
    HEALTH_WARN Reduced data availability: 1 pg inactive, 1 pg peering" in
    cluster log

[5239914]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:26:34
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239914/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239914/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=66fc32e021c64a4d852ad7d7f9487edc

    Command failed (s3 tests against rgw) on smithi174 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239916]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:25:54
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239916/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239916/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=7c76f10c40844b12921756c9ef7ca620

    Command failed (s3 tests against rgw) on smithi003 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239917]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:32:35
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239917/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239917/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=a08b6850f7a340f1a8acad350b9fd802

    Command failed (s3 tests against rgw) on smithi167 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239920]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:31:04
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239920/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239920/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=542a86dca23d47b88d89c9487b7be4dd

    Command failed (s3 tests against rgw) on smithi026 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239921]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:26:23
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239921/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239921/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=af08e2c372ce47e0a0e1b631e1f8acd6

    Command failed (s3 tests against rgw) on smithi150 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239922]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:30:28
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239922/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239922/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=dd90faf5ad0b47fc8316c9fb13b2dbb4

    Command failed (s3 tests against rgw) on smithi051 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239923]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:26:13
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239923/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239923/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=393c6f672721408e87be9422a097363a

    Command failed (s3 tests against rgw) on smithi069 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239924]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:29:38
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239924/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239924/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=3f5f7e54867048a2bbc7af50976a1cac

    Command failed (s3 tests against rgw) on smithi171 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239925]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:30:21
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239925/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239925/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=9eaedbbacbfe491eb6da0b0a575b1096

    Command failed (s3 tests against rgw) on smithi012 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239926]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:41:51
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239926/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239926/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=07209cbd445b4558a4ed952884330900

    "2020-07-19T14:00:00.000170+0000 mon.a (mon.0) 149 : cluster [WRN] overall
    HEALTH_WARN Reduced data availability: 1 pg inactive, 1 pg peering" in
    cluster log

[5239927]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:29:03
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239927/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239927/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=52c6bd1d70f3474c9e34a73e30d0b583

    Command failed (s3 tests against rgw) on smithi201 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239928]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:27:17
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239928/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239928/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=970686f4dd804fa2851184e0fbf09b5f

    Command failed (s3 tests against rgw) on smithi036 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"

[5239930]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:28:29
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239930/
log:    http://qa-proxy.ceph.com/teuthology/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239930/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=b3a17a216f314a46aad735203580acab

    Command failed (s3 tests against rgw) on smithi027 with status 1:
    "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt
    /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!
    fails_strict_rfc2616,!s3select,!fails_with_subdomain'"



Pass
=================================================================
[5239883] rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:29:09
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239883/

[5239885] rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:27:16
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239885/

[5239889] rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:30:25
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239889/

[5239893] rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:29:39
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239893/

[5239894] rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   00:26:34
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239894/

[5239898] rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   00:33:28
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239898/

[5239912] rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   00:28:56
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239912/

[5239915] rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:30:56
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239915/

[5239918] rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   00:28:03
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239918/

[5239919] rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:26:43
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239919/

[5239929] rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:29:50
info:   http://pulpito.ceph.com/gsalomon-2020-07-19_13:18:04-rgw:verify-wip-validate-test-setup-distro-basic-smithi/5239929/