2022-05-19T18:50:33.814 INFO:root:teuthology version: 1.1.1.dev333+gb018fa01 2022-05-19T18:50:33.817 INFO:teuthology.results:subset: '111/120000' 2022-05-19T18:50:33.817 INFO:teuthology.results:seed: '4894' 2022-05-19T19:17:12.323 INFO:root:teuthology version: 1.1.1.dev333+gb018fa01 2022-05-19T19:17:12.337 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-05-19T22:00:25.700 INFO:teuthology.results:Tests finished! gathering results... 2022-05-19T22:00:28.399 INFO:scrape:Found 40 jobs 2022-05-19T22:00:29.406 INFO:scrape:Found 3 distinct failure reasons 2022-05-19T22:00:29.407 INFO:scrape:Failure: Command failed on smithi139 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test_account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|.*test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_services_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*test_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticweb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.*test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds|.*test_object_services.ObjectTest.test_create_object_with_transfer_encoding'" 2022-05-19T22:00:29.407 INFO:scrape:1 jobs: ['6841484'] 2022-05-19T22:00:29.408 INFO:scrape:suites: ['frontend/beast', 'overrides', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}'] 2022-05-19T22:00:29.409 INFO:scrape: 2022-05-19T22:00:29.409 INFO:scrape:Failure: rgw multisite test failures 2022-05-19T22:00:29.410 INFO:scrape:1 jobs: ['6841480'] 2022-05-19T22:00:29.410 INFO:scrape:suites: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'rgw/multisite/{clusters', 'supported-random-distro$/{centos_8}', 'tasks/test_multi}'] 2022-05-19T22:00:29.411 INFO:scrape: 2022-05-19T22:00:29.411 INFO:scrape:Failure: Cannot create secret 2022-05-19T22:00:29.412 INFO:scrape:1 jobs: ['6841476'] 2022-05-19T22:00:29.412 INFO:scrape:suites: ['1-ceph-install/install', '2-kms/barbican', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability', 'rgw/crypt/{0-cluster/fixed-1', 'supported-random-distro$/{centos_8}}'] 2022-05-19T22:00:29.413 INFO:scrape: 2022-05-19T22:00:29.416 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/ failed: 3 dead: 0 running: 0 waiting: 0 queued: 0 passed: 37 Fail ================================================================= [6841476] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:15:55 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841476/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841476/ sentry: https://sentry.ceph.com/organizations/ceph/?query=db86d339492a4a208c776d8e00aa8f1c Cannot create secret [6841480] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{centos_8} tasks/test_multi} ----------------------------------------------------------------- time: 00:48:27 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841480/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841480/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7e1cdc2abb6141a1a8a97dc3795a903d rgw multisite test failures [6841484] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:22:46 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841484/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841484/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7897e8770ab7478f821e020630e35181 Command failed on smithi139 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test _account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|. *test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_serv ices_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*tes t_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticw eb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.* test_object_services.PublicObjectTest.test_access_public_container_object_w ithout_using_creds|.*test_object_services.ObjectTest.test_create_object_wit h_transfer_encoding'" Pass ================================================================= [6841477] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} time: 00:16:28 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841477/ [6841478] rgw/lifecycle/{cluster overrides supported-random-distro$/{ubuntu_latest} tasks/rgw_s3tests} time: 00:23:24 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841478/ [6841479] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload ubuntu_latest} time: 00:09:45 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841479/ [6841481] rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} time: 00:31:25 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841481/ [6841482] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:23:44 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841482/ [6841483] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast supported-random-distro$/{rhel_8} tasks/{0-install first ststests}} time: 00:40:51 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841483/ [6841485] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:13:49 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841485/ [6841486] rgw/tools/{centos_latest cluster tasks} time: 02:50:58 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841486/ [6841487] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:27:43 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841487/ [6841488] rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest} time: 00:17:28 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841488/ [6841489] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:12:28 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841489/ [6841490] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} time: 00:15:30 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841490/ [6841491] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_ragweed ubuntu_latest} time: 00:09:00 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841491/ [6841492] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{rhel_8}} time: 00:23:49 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841492/ [6841493] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:19:48 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841493/ [6841494] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 01:46:30 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841494/ [6841495] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} time: 00:15:11 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841495/ [6841496] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_s3tests ubuntu_latest} time: 00:14:23 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841496/ [6841497] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{rhel_8}} time: 00:24:15 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841497/ [6841498] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:13:06 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841498/ [6841499] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} time: 00:25:35 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841499/ [6841500] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} time: 00:15:35 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841500/ [6841501] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_user_quota ubuntu_latest} time: 00:09:29 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841501/ [6841502] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{ubuntu_latest} tasks/test_multi} time: 00:29:54 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841502/ [6841503] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_latest}} time: 00:11:47 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841503/ [6841504] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:14:38 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841504/ [6841505] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:27:57 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841505/ [6841506] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:13:45 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841506/ [6841507] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} time: 00:25:31 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841507/ [6841508] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_bucket_quota ubuntu_latest} time: 00:10:52 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841508/ [6841509] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_8}} time: 00:13:38 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841509/ [6841510] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:17:00 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841510/ [6841511] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 01:48:40 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841511/ [6841512] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} time: 00:26:11 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841512/ [6841513] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload ubuntu_latest} time: 00:09:02 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841513/ [6841514] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{centos_8}} time: 00:13:20 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841514/ [6841515] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:12:43 info: http://pulpito.ceph.com/yuriw-2022-05-19_18:49:16-rgw-wip-yuri4-testing-2022-05-19-0831-quincy-distro-default-smithi/6841515/