2021-02-17T09:17:05.737 INFO:root:teuthology version: 1.1.0-071a4df2 2021-02-17T09:17:05.738 INFO:teuthology.results:seed: '6036' 2021-02-17T09:19:17.153 INFO:root:teuthology version: 1.1.0-071a4df2 2021-02-17T09:19:17.159 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-02-17T10:05:17.978 INFO:teuthology.results:Tests finished! gathering results... 2021-02-17T10:05:18.281 INFO:scrape:Found 6 jobs 2021-02-17T10:05:18.440 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890483/teuthology.log matches 2021-02-17T10:05:18.441 WARNING:scrape:Not-understood crash indication 2021-02-17T10:05:18.496 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890483/teuthology.log matches 2021-02-17T10:05:18.497 WARNING:scrape:Not-understood crash indication 2021-02-17T10:05:18.558 WARNING:scrape:Not-understood crash indication Binary file /home/teuthworker/archive/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890483/teuthology.log matches 2021-02-17T10:05:18.560 WARNING:scrape:Not-understood crash indication 2021-02-17T10:05:19.802 INFO:scrape:Found 2 distinct failure reasons 2021-02-17T10:05:19.803 INFO:scrape:Failure: Command failed on smithi142 with status 1: 'cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test' 2021-02-17T10:05:19.804 INFO:scrape:1 jobs: ['5890483'] 2021-02-17T10:05:19.804 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'frontend/beast', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw-datacache', 'rgw:verify/{centos_latest', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{0-install', 'validater/lockdep}'] 2021-02-17T10:05:19.805 INFO:scrape: 2021-02-17T10:05:19.805 INFO:scrape:Failure: Command failed (s3 tests against rgw) on smithi002 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!s3select,!test_of_sts,!webidentity_test,!fails_with_subdomain'" 2021-02-17T10:05:19.805 INFO:scrape:3 jobs: ['5890481', '5890485', '5890489'] 2021-02-17T10:05:19.806 INFO:scrape:suites intersection: ['cls', 'clusters/fixed-2', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw-datacache', 'rgw:verify/{centos_latest', 's3tests-java', 's3tests}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/valgrind}'] 2021-02-17T10:05:19.806 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw-datacache', 'rgw:verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/valgrind}'] 2021-02-17T10:05:19.807 INFO:scrape: 2021-02-17T10:05:19.808 INFO:teuthology.results:Sending results to mkogan@redhat.com: Test Run: mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 2 Fail ================================================================= [5890481] rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:32:01 info: http://pulpito.ceph.com/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890481/ log: http://qa-proxy.ceph.com/teuthology/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890481/ sentry: https://sentry.ceph.com/organizations/ceph/?query=def219b6fc2041c797f09519e58829df Command failed (s3 tests against rgw) on smithi002 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!s3select,!test_of_sts,!webidentity_test,!fails_with_s ubdomain'" [5890483] rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:38:30 info: http://pulpito.ceph.com/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890483/ log: http://qa-proxy.ceph.com/teuthology/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890483/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a3c55d1dfae442f0bb03fd83024d49f1 Command failed on smithi142 with status 1: 'cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test' [5890485] rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:31:48 info: http://pulpito.ceph.com/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890485/ log: http://qa-proxy.ceph.com/teuthology/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890485/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cafb682e1ba7475297d1e54fdded6eea Command failed (s3 tests against rgw) on smithi061 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!s3select,!test_of_sts,!webidentity_test,!fails_with_s ubdomain'" [5890489] rgw:verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:31:29 info: http://pulpito.ceph.com/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890489/ log: http://qa-proxy.ceph.com/teuthology/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890489/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9e4d790d024c48b8ac786710a96ea71b Command failed (s3 tests against rgw) on smithi139 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!s3select,!test_of_sts,!webidentity_test,!fails_with_s ubdomain'" Pass ================================================================= [5890480] rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep} time: 00:31:37 info: http://pulpito.ceph.com/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890480/ [5890487] rgw:verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard rgw-datacache s3tests-java s3tests} validater/lockdep} time: 00:31:16 info: http://pulpito.ceph.com/mkogan-2021-02-17_09:07:41-rgw:verify-wip-rgw-d3n-cache-upstreaming_i39-distro-basic-smithi/5890487/