2021-01-10T13:19:52.301 INFO:root:teuthology version: 1.0.0-77817aa4
2021-01-10T13:19:52.303 INFO:teuthology.results:seed: '8969'
2021-01-10T13:47:22.737 INFO:root:teuthology version: 1.0.0-77817aa4
2021-01-10T13:47:22.768 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-01-10T14:40:23.765 INFO:teuthology.results:Tests finished! gathering results...
2021-01-10T14:40:37.921 INFO:scrape:Found 48 jobs
2021-01-10T14:40:44.475 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772868/teuthology.log matches
2021-01-10T14:40:44.476 WARNING:scrape:Not-understood crash indication 
2021-01-10T14:40:44.549 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772868/teuthology.log matches
2021-01-10T14:40:44.550 WARNING:scrape:Not-understood crash indication 
2021-01-10T14:40:49.972 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772861/teuthology.log matches
2021-01-10T14:40:49.974 WARNING:scrape:Not-understood crash indication 
2021-01-10T14:40:50.047 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772861/teuthology.log matches
2021-01-10T14:40:50.048 WARNING:scrape:Not-understood crash indication 
2021-01-10T14:41:04.542 INFO:scrape:Found 3 distinct failure reasons
2021-01-10T14:41:04.543 INFO:scrape:Failure: Command failed (ragweed tests against rgw) on smithi140 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"
2021-01-10T14:41:04.544 INFO:scrape:27 jobs: ['5772869', '5772856', '5772875', '5772860', '5772849', '5772859', '5772876', '5772878', '5772840', '5772857', '5772870', '5772838', '5772836', '5772850', '5772842', '5772848', '5772880', '5772871', '5772874', '5772864', '5772866', '5772862', '5772846', '5772872', '5772853', '5772865', '5772837']
2021-01-10T14:41:04.545 INFO:scrape:suites intersection: ['cls', 'clusters/fixed-2', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw-datacache', 'rgw:verify/{centos_latest', 's3tests-java', 's3tests}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install']
2021-01-10T14:41:04.545 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw-datacache', 'rgw:verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}']
2021-01-10T14:41:04.546 INFO:scrape:
2021-01-10T14:41:04.546 INFO:scrape:Failure: Command failed on smithi066 with status 1: 'cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test'
2021-01-10T14:41:04.547 INFO:scrape:10 jobs: ['5772844', '5772852', '5772833', '5772868', '5772861', '5772835', '5772873', '5772854', '5772845', '5772847']
2021-01-10T14:41:04.548 INFO:scrape:suites intersection: ['cls', 'clusters/fixed-2', 'msgr-failures/few', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw-datacache', 'rgw:verify/{centos_latest', 's3tests-java', 's3tests}', 'striping$/{stripe-equals-chunk}', 'tasks/{0-install']
2021-01-10T14:41:04.548 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw-datacache', 'rgw:verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}']
2021-01-10T14:41:04.549 INFO:scrape:
2021-01-10T14:41:04.549 INFO:scrape:Failure: Command failed on smithi170 with status 75: 's3cmd --access_key=0555b35654ad1656d704 --secret_key=h7GhxuBLTrlhVUyxSPUKUV7r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q== --config=/home/ubuntu/cephtest/archive/s3cfg --host=smithi170.front.sepia.ceph.com:443 mb s3://btk'
2021-01-10T14:41:04.550 INFO:scrape:11 jobs: ['5772851', '5772839', '5772834', '5772877', '5772863', '5772841', '5772843', '5772879', '5772855', '5772867', '5772858']
2021-01-10T14:41:04.550 INFO:scrape:suites intersection: ['cls', 'clusters/fixed-2', 'msgr-failures/few', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw-datacache', 'rgw:verify/{centos_latest', 's3tests-java', 's3tests}', 'striping$/{stripe-equals-chunk}', 'tasks/{0-install']
2021-01-10T14:41:04.551 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw-datacache', 'rgw:verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}']
2021-01-10T14:41:04.551 INFO:scrape:
2021-01-10T14:41:04.554 INFO:teuthology.results:Sending results to mkogan@redhat.com: Test Run: mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/
failed:  48
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[5772833]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:26:27
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772833/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772833/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9d0854dc775d4b7781a722627c22e25e

    Command failed on smithi078 with status 1: 'cd
    /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean
    test --rerun-tasks --no-build-cache --tests ObjectTest'

[5772834]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:28:02
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772834/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772834/
sentry: https://sentry.ceph.com/organizations/ceph/?query=891f47637b5a4871ae07d66bb3afb85d

    Command failed on smithi033 with status 75: 's3cmd
    --access_key=0555b35654ad1656d704
    --secret_key=h7GhxuBLTrlhVUyxSPUKUV7r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==
    --config=/home/ubuntu/cephtest/archive/s3cfg
    --host=smithi033.front.sepia.ceph.com:443 mb s3://btk'

[5772835]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:24:22
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772835/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772835/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2110607fb7b249d5ab83f2144e4052be

    Command failed on smithi067 with status 1: 'cd
    /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean
    test --rerun-tasks --no-build-cache --tests AWS4Test'

[5772836]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:22:08
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772836/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772836/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bc974d3b08d64bab8951859f241d65a0

    Command failed (ragweed tests against rgw) on smithi044 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772837]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:21:22
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772837/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772837/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8f35694c0bfd4362a1dee6a77b9144c8

    Command failed (ragweed tests against rgw) on smithi074 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772838]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:20:57
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772838/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772838/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a62f5c7ed85748d280065b218deae168

    Command failed (ragweed tests against rgw) on smithi180 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772839]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:28:57
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772839/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772839/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7573e779590944319f59a711b55426e2

    Command failed on smithi125 with status 75: 's3cmd
    --access_key=0555b35654ad1656d704
    --secret_key=h7GhxuBLTrlhVUyxSPUKUV7r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==
    --config=/home/ubuntu/cephtest/archive/s3cfg
    --host=smithi125.front.sepia.ceph.com:443 mb s3://btk'

[5772840]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:20:32
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772840/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772840/
sentry: https://sentry.ceph.com/organizations/ceph/?query=52f68b7a38f64f61ab087359f2200ff9

    Command failed (ragweed tests against rgw) on smithi106 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772841]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:29:50
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772841/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772841/
sentry: https://sentry.ceph.com/organizations/ceph/?query=946ffe9b249c41ceb63d5c4f67e96403

    Command failed on smithi006 with status 75: 's3cmd
    --access_key=0555b35654ad1656d704
    --secret_key=h7GhxuBLTrlhVUyxSPUKUV7r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==
    --config=/home/ubuntu/cephtest/archive/s3cfg
    --host=smithi006.front.sepia.ceph.com:443 mb s3://btk'

[5772842]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:20:37
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772842/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772842/
sentry: https://sentry.ceph.com/organizations/ceph/?query=749d612968e8406589f9b5b01b133cb7

    Command failed (ragweed tests against rgw) on smithi115 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772843]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:28:47
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772843/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772843/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f509d0e6669d4a07bce6b4fb50426d25

    Command failed on smithi185 with status 75: 's3cmd
    --access_key=0555b35654ad1656d704
    --secret_key=h7GhxuBLTrlhVUyxSPUKUV7r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==
    --config=/home/ubuntu/cephtest/archive/s3cfg
    --host=smithi185.front.sepia.ceph.com:443 mb s3://btk'

[5772844]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:23:11
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772844/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772844/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8a813b8503cf4e918ac2beb22f92f5a7

    Command failed on smithi066 with status 1: 'cd
    /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean
    test --rerun-tasks --no-build-cache --tests AWS4Test'

[5772845]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:23:31
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772845/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772845/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5e5a765044ea480c84e6780054cd0571

    Command failed on smithi055 with status 1: 'cd
    /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean
    test --rerun-tasks --no-build-cache --tests AWS4Test'

[5772846]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:22:47
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772846/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772846/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a6c969ccb9464f22b004213e90339636

    Command failed (ragweed tests against rgw) on smithi149 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772847]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:23:23
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772847/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772847/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b13f6f387cb144ba84a7527683ba2407

    Command failed on smithi142 with status 1: 'cd
    /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean
    test --rerun-tasks --no-build-cache --tests AWS4Test'

[5772848]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:22:07
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772848/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772848/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fa8368b3fd6540a9964333353c16c932

    Command failed (ragweed tests against rgw) on smithi199 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772849]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:20:28
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772849/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772849/
sentry: https://sentry.ceph.com/organizations/ceph/?query=609236fbc6b549de885c41a7a6aafd54

    Command failed (ragweed tests against rgw) on smithi200 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772850]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:23:01
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772850/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772850/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f4008bcceb194a159dae1284ef9d0e1f

    Command failed (ragweed tests against rgw) on smithi014 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772851]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:27:25
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772851/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772851/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ac655f95bcee4db88c573f1159178f04

    Command failed on smithi170 with status 75: 's3cmd
    --access_key=0555b35654ad1656d704
    --secret_key=h7GhxuBLTrlhVUyxSPUKUV7r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==
    --config=/home/ubuntu/cephtest/archive/s3cfg
    --host=smithi170.front.sepia.ceph.com:443 mb s3://btk'

[5772852]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:24:19
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772852/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772852/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6b9a0440bea14ad191192ced3aa0e817

    Command failed on smithi073 with status 1: 'cd
    /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean
    test --rerun-tasks --no-build-cache --tests AWS4Test'

[5772853]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:20:55
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772853/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772853/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0aa1525d61b7472bb89d35ae59c60037

    Command failed (ragweed tests against rgw) on smithi099 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772854]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:24:53
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772854/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772854/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c0729946e8674ef0bf8ef5a64477adc9

    Command failed on smithi123 with status 1: 'cd
    /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean
    test --rerun-tasks --no-build-cache --tests AWS4Test'

[5772855]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:27:50
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772855/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772855/
sentry: https://sentry.ceph.com/organizations/ceph/?query=98db45aaeadb435ab2fe39cf6c2e8473

    Command failed on smithi144 with status 75: 's3cmd
    --access_key=0555b35654ad1656d704
    --secret_key=h7GhxuBLTrlhVUyxSPUKUV7r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==
    --config=/home/ubuntu/cephtest/archive/s3cfg
    --host=smithi144.front.sepia.ceph.com:443 mb s3://btk'

[5772856]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:25:34
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772856/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772856/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7454ebcabed54dd8a91f72496a00ecde

    Command failed (ragweed tests against rgw) on smithi042 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772857]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:21:09
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772857/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772857/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fd8f4af7d1f34879b0aca25c5c9fd809

    Command failed (ragweed tests against rgw) on smithi118 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772858]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:27:48
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772858/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772858/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3b8011ae21914b999c4279c49a500e9b

    Command failed on smithi155 with status 75: 's3cmd
    --access_key=0555b35654ad1656d704
    --secret_key=h7GhxuBLTrlhVUyxSPUKUV7r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==
    --config=/home/ubuntu/cephtest/archive/s3cfg
    --host=smithi155.front.sepia.ceph.com:443 mb s3://btk'

[5772859]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:21:13
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772859/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772859/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d1ab522c5e424275a1d1bbf3e77d4b88

    Command failed (ragweed tests against rgw) on smithi137 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772860]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:21:55
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772860/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772860/
sentry: https://sentry.ceph.com/organizations/ceph/?query=36f6df64894f409ea40bfdb77ae4b9ad

    Command failed (ragweed tests against rgw) on smithi169 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772861]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:55:32
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772861/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772861/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9fd0d2d8ac774b09a979c5958f8bcb1c

    Command failed on smithi146 with status 1: 'cd
    /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean
    test --rerun-tasks --no-build-cache --tests AWS4Test'

[5772862]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:21:53
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772862/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772862/
sentry: https://sentry.ceph.com/organizations/ceph/?query=419a71d1fb314271a071ea6c3f8465cf

    Command failed (ragweed tests against rgw) on smithi156 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772863]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:29:54
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772863/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772863/
sentry: https://sentry.ceph.com/organizations/ceph/?query=14ae89470a504594914e291169fdb9bd

    Command failed on smithi035 with status 75: 's3cmd
    --access_key=0555b35654ad1656d704
    --secret_key=h7GhxuBLTrlhVUyxSPUKUV7r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==
    --config=/home/ubuntu/cephtest/archive/s3cfg
    --host=smithi035.front.sepia.ceph.com:443 mb s3://btk'

[5772864]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:23:13
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772864/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772864/
sentry: https://sentry.ceph.com/organizations/ceph/?query=607a602e54d84081ac5d34666853397a

    Command failed (ragweed tests against rgw) on smithi100 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772865]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:22:02
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772865/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772865/
sentry: https://sentry.ceph.com/organizations/ceph/?query=dc64361245fa4eb9afbb3b9d7a3e73cf

    Command failed (ragweed tests against rgw) on smithi184 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772866]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:20:58
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772866/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772866/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7e862270aa164921b156d25879fa97bc

    Command failed (ragweed tests against rgw) on smithi084 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772867]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:29:52
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772867/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772867/
sentry: https://sentry.ceph.com/organizations/ceph/?query=073ac19ae1914a538612461f2785d0bf

    Command failed on smithi139 with status 75: 's3cmd
    --access_key=0555b35654ad1656d704
    --secret_key=h7GhxuBLTrlhVUyxSPUKUV7r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==
    --config=/home/ubuntu/cephtest/archive/s3cfg
    --host=smithi139.front.sepia.ceph.com:443 mb s3://btk'

[5772868]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:54:51
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772868/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772868/
sentry: https://sentry.ceph.com/organizations/ceph/?query=33e8a50ee616477997301a920ee1436c

    Command failed on smithi145 with status 1: 'cd
    /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean
    test --rerun-tasks --no-build-cache --tests AWS4Test'

[5772869]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:20:20
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772869/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772869/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e788afd1bb2144e580b0f2fa4b50db8e

    Command failed (ragweed tests against rgw) on smithi140 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772870]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:22:26
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772870/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772870/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4cbf57fe675b48cd8e9bc33c924fbab7

    Command failed (ragweed tests against rgw) on smithi089 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772871]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:22:41
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772871/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772871/
sentry: https://sentry.ceph.com/organizations/ceph/?query=32e58de9063b418b824e8cd656b35dd9

    Command failed (ragweed tests against rgw) on smithi087 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772872]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:25:23
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772872/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772872/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cd3b30c63c8347fa9401ddf35c2558dc

    Command failed (ragweed tests against rgw) on smithi068 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772873]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:21:55
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772873/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772873/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ddf4619aa21743a1978bf9e05b895108

    Command failed on smithi116 with status 1: 'cd
    /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean
    test --rerun-tasks --no-build-cache --tests AWS4Test'

[5772874]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:24:42
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772874/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772874/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f0e14eb0cf154706af10c505335ac257

    Command failed (ragweed tests against rgw) on smithi186 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772875]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:21:32
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772875/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772875/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9756e93c74a745b9b477f98971f822a0

    Command failed (ragweed tests against rgw) on smithi195 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772876]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:21:28
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772876/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772876/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4273416bf05d4514bd3de05e092add32

    Command failed (ragweed tests against rgw) on smithi171 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772877]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:27:29
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772877/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772877/
sentry: https://sentry.ceph.com/organizations/ceph/?query=459924e9d1014d67a93a70d955e83801

    Command failed on smithi086 with status 75: 's3cmd
    --access_key=0555b35654ad1656d704
    --secret_key=h7GhxuBLTrlhVUyxSPUKUV7r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==
    --config=/home/ubuntu/cephtest/archive/s3cfg
    --host=smithi086.front.sepia.ceph.com:443 mb s3://btk'

[5772878]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:21:50
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772878/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772878/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2f1e704123a44499b932c036022c91fc

    Command failed (ragweed tests against rgw) on smithi201 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"

[5772879]  rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:27:19
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772879/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772879/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e491206d137f4fb4ad932f68078a9347

    Command failed on smithi122 with status 75: 's3cmd
    --access_key=0555b35654ad1656d704
    --secret_key=h7GhxuBLTrlhVUyxSPUKUV7r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==
    --config=/home/ubuntu/cephtest/archive/s3cfg
    --host=smithi122.front.sepia.ceph.com:443 mb s3://btk'

[5772880]  rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:22:15
info:   http://pulpito.ceph.com/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772880/
log:    http://qa-proxy.ceph.com/teuthology/mkogan-2021-01-10_12:41:47-rgw:verify-wip-rgw-d3n-cache-upstreaming_i33-distro-basic-smithi/5772880/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bc232c887e8b4de29077de66f4174619

    Command failed (ragweed tests against rgw) on smithi038 with status 1:
    "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf
    RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg
    /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w
    /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"