2022-03-12T20:02:07.458 INFO:root:teuthology version: 1.1.0-fbfc7b38 2022-03-12T20:02:07.459 INFO:teuthology.results:subset: '111/120000' 2022-03-12T20:02:07.459 INFO:teuthology.results:seed: '7231' 2022-03-12T20:21:41.600 INFO:root:teuthology version: 1.1.0-fbfc7b38 2022-03-12T20:21:41.632 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-03-12T23:04:54.344 INFO:teuthology.results:Tests finished! gathering results... 2022-03-12T23:04:57.280 INFO:scrape:Found 40 jobs 2022-03-12T23:04:58.249 INFO:scrape:Found 3 distinct failure reasons 2022-03-12T23:04:58.250 INFO:scrape:Failure: Command failed on smithi036 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test_account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|.*test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_services_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*test_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticweb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.*test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds|.*test_object_services.ObjectTest.test_create_object_with_transfer_encoding'" 2022-03-12T23:04:58.250 INFO:scrape:1 jobs: ['6732650'] 2022-03-12T23:04:58.251 INFO:scrape:suites: ['frontend/beast', 'overrides', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}'] 2022-03-12T23:04:58.251 INFO:scrape: 2022-03-12T23:04:58.252 INFO:scrape:Failure: rgw multisite test failures 2022-03-12T23:04:58.252 INFO:scrape:1 jobs: ['6732646'] 2022-03-12T23:04:58.253 INFO:scrape:suites: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'rgw/multisite/{clusters', 'supported-random-distro$/{rhel_8}', 'tasks/test_multi}'] 2022-03-12T23:04:58.253 INFO:scrape: 2022-03-12T23:04:58.254 INFO:scrape:Failure: Cannot create secret 2022-03-12T23:04:58.254 INFO:scrape:1 jobs: ['6732642'] 2022-03-12T23:04:58.255 INFO:scrape:suites: ['1-ceph-install/install', '2-kms/barbican', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability', 'rgw/crypt/{0-cluster/fixed-1', 'supported-random-distro$/{centos_8}}'] 2022-03-12T23:04:58.255 INFO:scrape: 2022-03-12T23:04:58.257 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/ failed: 3 dead: 0 running: 0 waiting: 0 queued: 0 passed: 37 Fail ================================================================= [6732642] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:14:10 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732642/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732642/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e639270bee954701a2b3f99342585522 Cannot create secret [6732646] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{rhel_8} tasks/test_multi} ----------------------------------------------------------------- time: 00:59:28 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732646/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732646/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ec077a0af87e467f98eca09e6851506f rgw multisite test failures [6732650] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:21:20 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732650/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732650/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f9249d5f91c6495ab14d5c3da1fbddc2 Command failed on smithi036 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test _account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|. *test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_serv ices_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*tes t_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticw eb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.* test_object_services.PublicObjectTest.test_access_public_container_object_w ithout_using_creds|.*test_object_services.ObjectTest.test_create_object_wit h_transfer_encoding'" Pass ================================================================= [6732643] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} time: 00:17:21 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732643/ [6732644] rgw/lifecycle/{cluster overrides supported-random-distro$/{rhel_8} tasks/rgw_s3tests} time: 00:36:21 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732644/ [6732645] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload ubuntu_latest} time: 00:07:48 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732645/ [6732647] rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} time: 00:33:48 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732647/ [6732648] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:23:55 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732648/ [6732649] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast supported-random-distro$/{ubuntu_latest} tasks/{0-install first ststests}} time: 00:25:12 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732649/ [6732651] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:12:23 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732651/ [6732652] rgw/tools/{centos_latest cluster tasks} time: 02:49:07 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732652/ [6732653] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:25:57 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732653/ [6732654] rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest} time: 00:16:32 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732654/ [6732655] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:11:21 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732655/ [6732656] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} time: 00:28:44 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732656/ [6732657] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_ragweed ubuntu_latest} time: 00:07:22 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732657/ [6732658] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{centos_8}} time: 00:11:38 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732658/ [6732659] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:18:56 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732659/ [6732660] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 01:46:18 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732660/ [6732661] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{ubuntu_latest}} time: 00:11:12 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732661/ [6732662] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_s3tests ubuntu_latest} time: 00:12:35 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732662/ [6732663] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} time: 00:09:25 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732663/ [6732664] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:10:15 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732664/ [6732665] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} time: 00:25:39 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732665/ [6732666] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} time: 00:14:02 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732666/ [6732667] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_user_quota ubuntu_latest} time: 00:07:46 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732667/ [6732668] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_8} tasks/test_multi} time: 00:31:43 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732668/ [6732669] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:24:07 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732669/ [6732670] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:11:45 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732670/ [6732671] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:26:11 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732671/ [6732672] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:11:33 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732672/ [6732673] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{ubuntu_latest}} time: 00:11:21 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732673/ [6732674] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_bucket_quota ubuntu_latest} time: 00:09:03 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732674/ [6732675] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_8}} time: 00:11:58 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732675/ [6732676] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:15:20 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732676/ [6732677] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 01:40:34 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732677/ [6732678] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} time: 00:25:26 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732678/ [6732679] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload ubuntu_latest} time: 00:07:42 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732679/ [6732680] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{rhel_8}} time: 00:22:48 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732680/ [6732681] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:10:54 info: http://pulpito.ceph.com/yuriw-2022-03-12_00:05:28-rgw-master-distro-default-smithi/6732681/