2020-06-29T03:23:10.221 INFO:root:teuthology version: 1.0.0-22bcea5 2020-06-29T03:23:10.223 INFO:teuthology.results:seed: '8225' 2020-06-29T04:41:44.270 INFO:root:teuthology version: 1.0.0-22bcea5 2020-06-29T04:41:44.306 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-06-29T05:33:45.518 INFO:teuthology.results:Tests finished! gathering results... 2020-06-29T05:33:54.846 INFO:scrape:Found 107 jobs 2020-06-29T05:34:36.112 INFO:scrape:Found 7 distinct failure reasons 2020-06-29T05:34:36.114 INFO:scrape:Failure: Command failed (s3 tests against rgw) on smithi077 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain'" 2020-06-29T05:34:36.115 INFO:scrape:31 jobs 2020-06-29T05:34:36.116 INFO:scrape:suites intersection: ['cls', 'clusters/fixed-2', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 's3tests-java', 's3tests}', 'tasks/{0-install'] 2020-06-29T05:34:36.116 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}'] 2020-06-29T05:34:36.117 INFO:scrape: 2020-06-29T05:34:36.117 INFO:scrape:Failure: rgw multisite, test failures 2020-06-29T05:34:36.118 INFO:scrape:4 jobs: ['5187629', '5187657', '5187683', '5187710'] 2020-06-29T05:34:36.118 INFO:scrape:suites intersection: ['omap_limits', 'overrides', 'rgw/multisite/{clusters', 'tasks/test_multi', 'valgrind}'] 2020-06-29T05:34:36.118 INFO:scrape:suites union: ['frontend/beast', 'frontend/civetweb', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'tasks/test_multi', 'valgrind}'] 2020-06-29T05:34:36.119 INFO:scrape: 2020-06-29T05:34:36.120 INFO:scrape:Failure: Command failed on smithi013 with status 1: "cd /home/ubuntu/cephtest/keystone && source .tox/venv/bin/activate && openstack service create --os-username admin --os-password ADMIN --os-user-domain-id default --os-project-name admin --os-project-domain-id default --os-identity-api-version 3 --os-auth-url http://smithi013.front.sepia.ceph.com:35357/v3 --description 'Swift Service' --name swift object-store --debug" 2020-06-29T05:34:36.120 INFO:scrape:3 jobs: ['5187685', '5187631', '5187626'] 2020-06-29T05:34:36.120 INFO:scrape:suites intersection: [] 2020-06-29T05:34:36.121 INFO:scrape:suites union: ['1-ceph-install/install', '2-kms/barbican', '3-rgw/rgw', '4-tests/{s3tests}}', 'frontend/beast', 'frontend/civetweb', 'overrides', 'rgw/crypt/{0-cluster/fixed-1', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}'] 2020-06-29T05:34:36.121 INFO:scrape: 2020-06-29T05:34:36.122 INFO:scrape:Failure: "2020-06-29T04:38:31.606924+0000 osd.5 (osd.5) 3 : cluster [WRN] slow request osd_op(client.4144.0:1 2.2 2.7c339972 (undecoded) ondisk+read+known_if_redirected e19) initiated 2020-06-29T04:38:01.459531+0000 currently delayed" in cluster log 2020-06-29T05:34:36.122 INFO:scrape:2 jobs: ['5187704', '5187639'] 2020-06-29T05:34:36.123 INFO:scrape:suites intersection: ['cls', 'clusters/fixed-2', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/valgrind}'] 2020-06-29T05:34:36.123 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{0-install', 'validater/valgrind}'] 2020-06-29T05:34:36.123 INFO:scrape: 2020-06-29T05:34:36.124 INFO:scrape:Failure: "2020-06-29T05:00:00.000142+0000 mon.a (mon.0) 301 : cluster [WRN] overall HEALTH_WARN Reduced data availability: 1 pg inactive, 1 pg peering" in cluster log 2020-06-29T05:34:36.124 INFO:scrape:7 jobs: ['5187717', '5187637', '5187700', '5187692', '5187665', '5187726', '5187641'] 2020-06-29T05:34:36.125 INFO:scrape:suites intersection: ['overrides'] 2020-06-29T05:34:36.125 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/multifs/{clusters/fixed-2', 'rgw/verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/rgw_bucket_quota}', 'tasks/rgw_multipart_upload}', 'tasks/rgw_ragweed}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}'] 2020-06-29T05:34:36.126 INFO:scrape: 2020-06-29T05:34:36.126 INFO:scrape:Failure: Scrubbing terminated -- not all pgs were active and clean. 2020-06-29T05:34:36.126 INFO:scrape:5 jobs: ['5187662', '5187686', '5187632', '5187661', '5187711'] 2020-06-29T05:34:36.127 INFO:scrape:suites intersection: ['clusters/fixed-2'] 2020-06-29T05:34:36.127 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'install', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/thrash/{civetweb', 'rgw/verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{0-install', 'thrasher/default', 'thrashosds-health', 'validater/lockdep}', 'validater/valgrind}', 'workload/rgw_bucket_quota}', 'workload/rgw_s3tests}'] 2020-06-29T05:34:36.128 INFO:scrape: 2020-06-29T05:34:36.128 INFO:scrape:Failure: Command failed (workunit test rgw/test_rgw_orphan_list.sh) on smithi198 with status 12: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1169dee2592ec41777207fcf5742b5e936a100fc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_orphan_list.sh' 2020-06-29T05:34:36.128 INFO:scrape:1 jobs: ['5187633'] 2020-06-29T05:34:36.129 INFO:scrape:suites: ['cluster', 'rgw/tools/{centos_latest', 'tasks}'] 2020-06-29T05:34:36.129 INFO:scrape: 2020-06-29T05:34:36.135 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/ failed: 53 dead: 0 running: 0 waiting: 0 queued: 0 passed: 54 Fail ================================================================= [5187626] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests}} ----------------------------------------------------------------- time: 00:11:44 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187626/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187626/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=4d27f84082184cb1b58a22142a8d26dd Command failed on smithi084 with status 1: "cd /home/ubuntu/cephtest/keystone && source .tox/venv/bin/activate && openstack project create --os-username admin --os-password ADMIN --os-user- domain-id default --os-project-name admin --os-project-domain-id default --os-identity-api-version 3 --os-auth-url http://smithi084.front.sepia.ceph.com:35357/v3 --description 'Encryption Tenant' --domain default rgwcrypt --debug" [5187629] rgw/multisite/{clusters frontend/beast omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:46:47 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187629/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187629/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=7a343381a77d486cba46d003f06470fc rgw multisite, test failures [5187631] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:14:29 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187631/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187631/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=f2fccbf0b06f4ca1a5fe9ab4e5d3f5d0 Command failed on smithi050 with status 1: "cd /home/ubuntu/cephtest/keystone && source .tox/venv/bin/activate && openstack service create --os-username admin --os-password ADMIN --os-user- domain-id default --os-project-name admin --os-project-domain-id default --os-identity-api-version 3 --os-auth-url http://smithi050.front.sepia.ceph.com:35357/v3 --description 'Swift Service' --name swift object-store --debug" [5187632] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_bucket_quota} ----------------------------------------------------------------- time: 00:40:37 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187632/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187632/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=860992ec3c244b4693ca4196a727127d Scrubbing terminated -- not all pgs were active and clean. [5187633] rgw/tools/{centos_latest cluster tasks} ----------------------------------------------------------------- time: 00:15:51 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187633/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187633/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=179e451083b641beb02638801bd5699f Command failed (workunit test rgw/test_rgw_orphan_list.sh) on smithi198 with status 12: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1169dee2592ec41777207fcf5742b5e936a100fc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rgw/test_rgw_orphan_list.sh' [5187634] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:34:02 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187634/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187634/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=1bb2d278e04f4a7988657a139ea2b925 Command failed (s3 tests against rgw) on smithi158 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187636] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:37:37 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187636/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187636/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=2cd92ac5fd7f4dc18a0b683e08a51511 Command failed (s3 tests against rgw) on smithi109 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187637] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_multipart_upload} ----------------------------------------------------------------- time: 00:26:52 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187637/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187637/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=45dbb4ea7c0f466c908501dffb5928f5 "2020-06-29T03:47:25.345255+0000 mon.b (mon.0) 167 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5187639] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:41:20 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187639/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187639/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=df7340b93e0c41f38c337f49be81a81b "2020-06-29T03:41:13.751323+0000 osd.7 (osd.7) 3 : cluster [WRN] slow request osd_op(client.4156.0:2 2.1 2.3e6a8311 (undecoded) ondisk+read+known_if_redirected e18) initiated 2020-06-29T03:40:43.630198+0000 currently delayed" in cluster log [5187641] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:40:20 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187641/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187641/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=613e6054474747fd9464b3ab963a1393 "2020-06-29T04:00:00.000134+0000 mon.a (mon.0) 247 : cluster [WRN] overall HEALTH_WARN Reduced data availability: 1 pg inactive, 1 pg peering" in cluster log [5187643] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:36:59 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187643/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187643/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=8fa53c5bd7944e1182a4ea2d11fe167a Command failed (s3 tests against rgw) on smithi137 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187645] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:40:09 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187645/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187645/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=0b098139bf2840a1a4ba279f8e4fab37 Command failed (s3 tests against rgw) on smithi050 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187650] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:34:00 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187650/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187650/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=eab9e62d305d436cb3a325b1ad7de6aa Command failed (s3 tests against rgw) on smithi139 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187654] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:33:29 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187654/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187654/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=45f2179d51614642b0ca3a8d57c93651 Command failed (s3 tests against rgw) on smithi133 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187657] rgw/multisite/{clusters frontend/civetweb omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:31:45 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187657/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187657/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=f7d028b0c5324b528dd2d30221739f3c rgw multisite, test failures [5187661] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:41:32 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187661/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187661/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=5274e129bb174c7090336139bbdaf8f1 Scrubbing terminated -- not all pgs were active and clean. [5187662] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:40:08 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187662/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187662/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=68d958e249354e8f868bed52d20a9836 Scrubbing terminated -- not all pgs were active and clean. [5187664] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:38:37 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187664/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187664/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=e244ecebd79947c99ccc1406a9659a2e Command failed (s3 tests against rgw) on smithi055 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187665] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_bucket_quota} ----------------------------------------------------------------- time: 00:27:33 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187665/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187665/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=1564a68ddba5430f9c34b75fad49da49 "2020-06-29T04:11:28.481084+0000 mon.b (mon.0) 112 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5187667] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:32:13 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187667/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187667/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=5751ba0e4a804e9eb72c2fe2aeb6d08d Command failed (s3 tests against rgw) on smithi057 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187671] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:33:41 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187671/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187671/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=865737759d2d4008b563891325e4a7e3 Command failed (s3 tests against rgw) on smithi043 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187672] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:35:46 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187672/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187672/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=3495ef8e42d94fc9805a830ee23b3be5 Command failed (s3 tests against rgw) on smithi148 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187674] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:33:24 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187674/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187674/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=794f6e02eb9e42aa96bf178009914a47 Command failed (s3 tests against rgw) on smithi077 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187676] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:34:52 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187676/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187676/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=03faba0c72524f42adbbe8306ff57a7e Command failed (s3 tests against rgw) on smithi061 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187677] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:33:55 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187677/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187677/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=63775c8d37ef462398c18a5332b3d824 Command failed (s3 tests against rgw) on smithi158 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187679] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:34:28 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187679/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187679/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=61523be77e8b46d79328659f389b019b Command failed (s3 tests against rgw) on smithi087 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187683] rgw/multisite/{clusters frontend/civetweb omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:41:56 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187683/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187683/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=89d591913d364eeeb6fee42ae5294702 rgw multisite pubsub, test failures [5187685] rgw/tempest/{clusters/fixed-1 frontend/civetweb overrides tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:12:48 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187685/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187685/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=52357cb146094c39b35d9802fcb1bf73 Command failed on smithi013 with status 1: "cd /home/ubuntu/cephtest/keystone && source .tox/venv/bin/activate && openstack service create --os-username admin --os-password ADMIN --os-user- domain-id default --os-project-name admin --os-project-domain-id default --os-identity-api-version 3 --os-auth-url http://smithi013.front.sepia.ceph.com:35357/v3 --description 'Swift Service' --name swift object-store --debug" [5187686] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_bucket_quota} ----------------------------------------------------------------- time: 00:40:46 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187686/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187686/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=541495f27f2e43149e087e584b1028ca Scrubbing terminated -- not all pgs were active and clean. [5187687] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:35:00 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187687/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187687/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=3bbf97e370a546c9a9ff3cf1a15ca82d Command failed (s3 tests against rgw) on smithi064 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187689] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:33:34 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187689/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187689/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=0a9e2893aaa44fcc81be146b4174f1d4 Command failed (s3 tests against rgw) on smithi008 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187691] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:35:20 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187691/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187691/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=9cd1ce54cef9450c9067e0aa20cbf9a7 Command failed (s3 tests against rgw) on smithi099 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187692] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:41:08 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187692/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187692/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=36c6c6aa316d4dea89a4a6696b32600f "2020-06-29T05:00:00.000139+0000 mon.b (mon.0) 114 : cluster [WRN] overall HEALTH_WARN Reduced data availability: 1 pg inactive, 1 pg peering" in cluster log [5187694] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:35:04 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187694/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187694/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=7aa282469f164f3a958d7e96d0cb6591 Command failed (s3 tests against rgw) on smithi036 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187696] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:35:04 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187696/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187696/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=80ee471c28804cf9b73bd05f3fc542ae Command failed (s3 tests against rgw) on smithi089 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187700] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:41:10 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187700/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187700/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=3e3eca0481d64f7388cbf0f94ea91b19 "2020-06-29T05:00:00.000117+0000 mon.b (mon.0) 115 : cluster [WRN] overall HEALTH_WARN Reduced data availability: 2 pgs inactive, 2 pgs peering" in cluster log [5187703] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:38:14 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187703/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187703/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=213da31f59d64ab6b02d5930543aa418 Command failed (s3 tests against rgw) on smithi078 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187704] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:41:18 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187704/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187704/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=563e71ba0c954ca18d03d0239779ff83 "2020-06-29T04:38:31.606924+0000 osd.5 (osd.5) 3 : cluster [WRN] slow request osd_op(client.4144.0:1 2.2 2.7c339972 (undecoded) ondisk+read+known_if_redirected e19) initiated 2020-06-29T04:38:01.459531+0000 currently delayed" in cluster log [5187706] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:38:27 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187706/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187706/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=962de2dbfea647aab2753ca86ecee23f Command failed (s3 tests against rgw) on smithi060 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187707] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:35:09 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187707/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187707/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=90847cc5b15e4e5aae3bfe8e3aa5f57d Command failed (s3 tests against rgw) on smithi040 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187710] rgw/multisite/{clusters frontend/beast omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:30:19 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187710/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187710/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=ec0d9a0e61a24944bf631bb0bbd5e0f3 rgw multisite, test failures [5187711] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_s3tests} ----------------------------------------------------------------- time: 00:40:12 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187711/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187711/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=74ca1845f30f46a0826e34ee5f63fa70 Scrubbing terminated -- not all pgs were active and clean. [5187712] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:33:12 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187712/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187712/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=117fc1e24caa459690cdd3bdb39b2a98 Command failed (s3 tests against rgw) on smithi017 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187714] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:36:30 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187714/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187714/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=73e804a05fba4ac2b445849f928a56db Command failed (s3 tests against rgw) on smithi002 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187717] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:41:45 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187717/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187717/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=0f3062cf018b4367877f850c573ed2f0 "2020-06-29T05:00:00.000142+0000 mon.a (mon.0) 301 : cluster [WRN] overall HEALTH_WARN Reduced data availability: 1 pg inactive, 1 pg peering" in cluster log [5187720] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:34:19 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187720/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187720/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=778fa2caab1441f1900a9b0498abc963 Command failed (s3 tests against rgw) on smithi047 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187721] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:35:34 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187721/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187721/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=6d80cb2113314749a3f2c7d2aa3b49d4 Command failed (s3 tests against rgw) on smithi057 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187725] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:36:04 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187725/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187725/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=1b4001997404477d8572dcbe940b43d2 Command failed (s3 tests against rgw) on smithi112 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187726] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_ragweed} ----------------------------------------------------------------- time: 00:26:35 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187726/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187726/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=e2845ced254b437285ca3e7ad3ea1ffe "2020-06-29T04:53:00.753759+0000 mon.a (mon.0) 178 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log [5187727] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:36:26 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187727/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187727/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=fe7a9e4dc72b41a48445e5329e2f7d5f Command failed (s3 tests against rgw) on smithi148 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187729] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:34:58 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187729/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187729/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=261c61ff54e64c72aee6a1f50de1acbf Command failed (s3 tests against rgw) on smithi039 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187730] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} ----------------------------------------------------------------- time: 00:36:41 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187730/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187730/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=d3be53b540bf4a63abe8e5ec7c536081 Command failed (s3 tests against rgw) on smithi085 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" [5187732] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 00:33:55 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187732/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187732/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=fb1706348eee4c61af9996cd16c8fd51 Command failed (s3 tests against rgw) on smithi025 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!fails_with_subdomain'" Pass ================================================================= [5187627] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} time: 00:16:13 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187627/ [5187628] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_bucket_quota} time: 00:13:07 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187628/ [5187630] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:14:11 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187630/ [5187635] rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest} time: 00:17:26 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187635/ [5187638] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:33:52 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187638/ [5187640] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_ragweed} time: 00:08:25 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187640/ [5187642] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_s3tests} time: 00:13:08 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187642/ [5187644] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_multipart_upload} time: 00:15:06 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187644/ [5187646] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_user_quota} time: 00:12:04 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187646/ [5187647] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 00:34:52 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187647/ [5187648] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_bucket_quota} time: 00:12:29 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187648/ [5187649] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_latest}} time: 00:09:41 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187649/ [5187651] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 00:39:30 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187651/ [5187652] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload} time: 00:12:05 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187652/ [5187653] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:36:59 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187653/ [5187655] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_ragweed} time: 00:08:13 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187655/ [5187656] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests}} time: 00:13:36 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187656/ [5187658] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_s3tests} time: 00:17:45 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187658/ [5187659] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:32:42 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187659/ [5187660] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_s3tests} time: 00:14:42 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187660/ [5187663] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_user_quota} time: 00:11:19 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187663/ [5187666] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} time: 00:09:55 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187666/ [5187668] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 00:38:08 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187668/ [5187669] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload} time: 00:11:27 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187669/ [5187670] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:14:40 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187670/ [5187673] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_ragweed} time: 00:09:04 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187673/ [5187675] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_s3tests} time: 00:13:05 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187675/ [5187678] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_user_quota} time: 00:11:41 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187678/ [5187680] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests}} time: 00:13:08 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187680/ [5187681] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} time: 00:14:38 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187681/ [5187682] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_bucket_quota} time: 00:12:46 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187682/ [5187684] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_latest}} time: 00:09:47 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187684/ [5187688] rgw/website/{clusters/fixed-2 frontend/civetweb http overrides tasks/s3tests-website ubuntu_latest} time: 00:17:18 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187688/ [5187690] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_multipart_upload} time: 00:12:08 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187690/ [5187693] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_ragweed} time: 00:08:26 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187693/ [5187695] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_s3tests} time: 00:14:23 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187695/ [5187697] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_multipart_upload} time: 00:15:40 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187697/ [5187698] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:35:59 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187698/ [5187699] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_user_quota} time: 00:11:27 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187699/ [5187701] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_bucket_quota} time: 00:13:20 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187701/ [5187702] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_latest}} time: 00:09:57 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187702/ [5187705] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload} time: 00:11:29 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187705/ [5187708] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_ragweed} time: 00:09:31 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187708/ [5187709] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests}} time: 00:13:07 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187709/ [5187713] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_s3tests} time: 00:13:53 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187713/ [5187715] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:33:33 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187715/ [5187716] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_user_quota} time: 00:11:55 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187716/ [5187718] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_bucket_quota} time: 00:12:59 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187718/ [5187719] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} time: 00:10:07 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187719/ [5187722] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_multipart_upload} time: 00:12:07 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187722/ [5187723] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:15:35 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187723/ [5187724] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:41:47 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187724/ [5187728] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_s3tests} time: 00:14:23 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187728/ [5187731] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_user_quota} time: 00:11:23 info: http://pulpito.ceph.com/teuthology-2020-06-29_03:05:03-rgw-master-distro-basic-smithi/5187731/