2022-07-28T21:46:59.102 INFO:root:teuthology version: 1.1.1.dev401+gdd7eb9d9 2022-07-28T21:46:59.104 INFO:teuthology.results:subset: '1/20' 2022-07-28T21:46:59.105 INFO:teuthology.results:seed: '831' 2022-07-28T22:24:30.614 INFO:root:teuthology version: 1.1.1.dev401+gdd7eb9d9 2022-07-28T22:24:30.645 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-07-29T01:45:47.167 INFO:teuthology.results:Tests finished! gathering results... 2022-07-29T01:45:50.382 INFO:scrape:Found 38 jobs 2022-07-29T01:45:53.178 INFO:scrape:Found 3 distinct failure reasons 2022-07-29T01:45:53.179 INFO:scrape:Failure: rgw multisite test failures 2022-07-29T01:45:53.180 INFO:scrape:2 jobs: ['6953290', '6953270'] 2022-07-29T01:45:53.180 INFO:scrape:suites intersection: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'rgw/multisite/{clusters', 'tasks/test_multi}'] 2022-07-29T01:45:53.181 INFO:scrape:suites union: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}'] 2022-07-29T01:45:53.181 INFO:scrape: 2022-07-29T01:45:53.182 INFO:scrape:Failure: Cannot create secret 2022-07-29T01:45:53.182 INFO:scrape:1 jobs: ['6953265'] 2022-07-29T01:45:53.183 INFO:scrape:suites: ['1-ceph-install/install', '2-kms/barbican', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability', 'rgw/crypt/{0-cluster/fixed-1', 's3tests-branch', 'supported-random-distro$/{rhel_8}}'] 2022-07-29T01:45:53.183 INFO:scrape: 2022-07-29T01:45:53.184 INFO:scrape:Failure: Command failed on smithi100 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test_account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|.*test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_services_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*test_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticweb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.*test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds|.*test_object_services.ObjectTest.test_create_object_with_transfer_encoding'" 2022-07-29T01:45:53.184 INFO:scrape:1 jobs: ['6953274'] 2022-07-29T01:45:53.185 INFO:scrape:suites: ['frontend/beast', 'overrides', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}'] 2022-07-29T01:45:53.185 INFO:scrape: 2022-07-29T01:45:53.187 INFO:teuthology.results:Sending results to ivancich@redhat.com: Test Run: ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 34 Fail ================================================================= [6953265] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:18:16 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953265/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953265/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c45fed1ebfb1407f85928ed8475219ee Cannot create secret [6953270] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{centos_8} tasks/test_multi} ----------------------------------------------------------------- time: 03:48:55 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953270/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953270/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ae001a9f22ae4fbfb3b238524968dbb2 rgw multisite test failures [6953274] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:23:40 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953274/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953274/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d89ebb840769444c951f107cb7e21d58 Command failed on smithi100 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test _account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|. *test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_serv ices_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*tes t_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticw eb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.* test_object_services.PublicObjectTest.test_access_public_container_object_w ithout_using_creds|.*test_object_services.ObjectTest.test_create_object_wit h_transfer_encoding'" [6953290] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{ubuntu_latest} tasks/test_multi} ----------------------------------------------------------------- time: 02:14:13 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953290/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953290/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5dd409a705c640fcb64aa71ed5622a4a rgw multisite test failures Pass ================================================================= [6953263] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:14:38 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953263/ [6953264] rgw/cloud-transition/{cluster overrides s3tests-branch supported-random-distro$/{centos_8} tasks/cloud_transition_s3tests} time: 00:32:39 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953264/ [6953266] rgw/dbstore/{cluster overrides s3tests-branch supported-random-distro$/{rhel_8} tasks/rgw_s3tests} time: 00:17:15 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953266/ [6953267] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} time: 00:17:49 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953267/ [6953268] rgw/lifecycle/{cluster overrides s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/rgw_s3tests} time: 00:24:00 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953268/ [6953269] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_multipart_upload ubuntu_latest} time: 00:11:00 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953269/ [6953271] rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} time: 00:33:56 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953271/ [6953272] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_8}} time: 00:13:07 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953272/ [6953273] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast s3tests-branch supported-random-distro$/{centos_8} tasks/{0-install first ststests}} time: 00:29:27 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953273/ [6953275] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:14:06 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953275/ [6953276] rgw/tools/{centos_latest cluster tasks} time: 02:51:33 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953276/ [6953277] rgw/website/{clusters/fixed-2 frontend/beast http overrides s3tests-branch tasks/s3tests-website ubuntu_latest} time: 00:18:06 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953277/ [6953278] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:13:01 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953278/ [6953279] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{centos_8}} time: 00:15:41 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953279/ [6953280] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec s3tests-branch tasks/rgw_ragweed ubuntu_latest} time: 00:09:46 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953280/ [6953281] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{rhel_8}} time: 00:14:08 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953281/ [6953282] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:20:50 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953282/ [6953283] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_latest}} time: 00:12:50 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953283/ [6953284] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_s3tests ubuntu_latest} time: 00:14:55 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953284/ [6953285] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} time: 00:11:06 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953285/ [6953286] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:12:44 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953286/ [6953287] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{centos_8}} time: 00:15:12 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953287/ [6953288] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} time: 00:15:26 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953288/ [6953289] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_user_quota ubuntu_latest} time: 00:09:52 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953289/ [6953291] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_8}} time: 00:13:03 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953291/ [6953292] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:14:12 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953292/ [6953293] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:13:22 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953293/ [6953294] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{centos_8}} time: 00:15:02 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953294/ [6953295] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_bucket_quota ubuntu_latest} time: 00:11:13 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953295/ [6953296] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_latest}} time: 00:11:28 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953296/ [6953297] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:17:08 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953297/ [6953298] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_latest}} time: 00:13:09 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953298/ [6953299] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_multipart_upload ubuntu_latest} time: 00:09:25 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953299/ [6953300] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} time: 00:11:08 info: http://pulpito.ceph.com/ivancich-2022-07-28_21:27:15-rgw-wip-eric-testing-1-distro-default-smithi/6953300/