2021-09-17T00:58:54.501 INFO:root:teuthology version: 1.1.0-4e1f59de 2021-09-17T00:58:54.503 INFO:teuthology.results:subset: '1/3' 2021-09-17T00:58:54.503 INFO:teuthology.results:seed: '831' 2021-09-17T01:06:08.197 INFO:root:teuthology version: 1.1.0-4e1f59de 2021-09-17T01:06:08.201 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-09-17T01:37:10.507 INFO:teuthology.results:Tests finished! gathering results... 2021-09-17T01:37:11.141 INFO:scrape:Found 16 jobs 2021-09-17T01:37:14.589 INFO:scrape:Found 1 distinct failure reasons 2021-09-17T01:37:14.590 INFO:scrape:Failure: Command failed (ragweed tests against rgw) on smithi018 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" 2021-09-17T01:37:14.590 INFO:scrape:16 jobs: ['6392811', '6392818', '6392808', '6392810', '6392820', '6392815', '6392813', '6392819', '6392817', '6392814', '6392807', '6392809', '6392822', '6392812', '6392816', '6392821'] 2021-09-17T01:37:14.591 INFO:scrape:suites intersection: ['centos_latest', 'clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw/verify/{0-install', 's3tests-java', 's3tests}', 'tasks/{cls'] 2021-09-17T01:37:14.592 INFO:scrape:suites union: ['centos_latest', 'clusters/fixed-2', 'datacache/no_datacache', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{cls', 'validater/lockdep}', 'validater/valgrind}'] 2021-09-17T01:37:14.592 INFO:scrape: 2021-09-17T01:37:14.593 INFO:teuthology.results:Sending results to ivancich@redhat.com: Test Run: ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/ failed: 16 dead: 0 running: 0 waiting: 0 queued: 0 passed: 0 Fail ================================================================= [6392807] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:21:01 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392807/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392807/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dad14dac412843a98b7c1a796cafd0bc Command failed (ragweed tests against rgw) on smithi068 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392808] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:21:59 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392808/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392808/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f2dcbd2ce13f491c9631dd3da4341480 Command failed (ragweed tests against rgw) on smithi099 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392809] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:20:51 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392809/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392809/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dc57d010cb294bfe8b0ca14c406caa8a Command failed (ragweed tests against rgw) on smithi052 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392810] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:21:28 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392810/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392810/ sentry: https://sentry.ceph.com/organizations/ceph/?query=54014bbf79fa4f19917175debd1533a4 Command failed (ragweed tests against rgw) on smithi145 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392811] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:20:16 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392811/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392811/ sentry: https://sentry.ceph.com/organizations/ceph/?query=99d18ac58a394070baa47ef6f503f1b1 Command failed (ragweed tests against rgw) on smithi018 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392812] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:21:55 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392812/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392812/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9106f30d39734e09a8080361425da3b2 Command failed (ragweed tests against rgw) on smithi058 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392813] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:21:07 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392813/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392813/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c40baa6fa6774aae903953a8fa57a670 Command failed (ragweed tests against rgw) on smithi027 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392814] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:21:19 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392814/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392814/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8e669e28cbd1470b85706bfd5fbe9427 Command failed (ragweed tests against rgw) on smithi162 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392815] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:22:07 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392815/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392815/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6b3fee103ebd46e8a5ae89a0e7217c8b Command failed (ragweed tests against rgw) on smithi050 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392816] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:21:15 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392816/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392816/ sentry: https://sentry.ceph.com/organizations/ceph/?query=768c893860594ae08586b895548fa0fc Command failed (ragweed tests against rgw) on smithi120 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392817] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:22:43 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392817/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392817/ sentry: https://sentry.ceph.com/organizations/ceph/?query=779b219d40df445bab058305608a6437 Command failed (ragweed tests against rgw) on smithi026 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392818] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:20:42 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392818/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392818/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4f24a9a971b74c9787311548466df319 Command failed (ragweed tests against rgw) on smithi148 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392819] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:21:36 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392819/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392819/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1b74fde84d9a46c193e157ead8677323 Command failed (ragweed tests against rgw) on smithi042 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392820] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:20:28 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392820/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392820/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b97b93601a15475da6fcbd993c1583ec Command failed (ragweed tests against rgw) on smithi007 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392821] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:22:16 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392821/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392821/ sentry: https://sentry.ceph.com/organizations/ceph/?query=71b8df91d57d4bc6989c448cad30b4ff Command failed (ragweed tests against rgw) on smithi049 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'" [6392822] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:19:53 info: http://pulpito.ceph.com/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392822/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2021-09-16_17:01:39-rgw-wip-eric-testing-1-distro-basic-smithi/6392822/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f6fbfc064d94489cab2ec705935231ce Command failed (ragweed tests against rgw) on smithi097 with status 1: "RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw'"