2022-03-05T22:34:35.051 INFO:root:teuthology version: 1.1.0-eeaeee38 2022-03-05T22:34:35.053 INFO:teuthology.results:subset: '111/120000' 2022-03-05T22:34:35.054 INFO:teuthology.results:seed: '1057' 2022-03-05T22:52:52.071 INFO:root:teuthology version: 1.1.0-eeaeee38 2022-03-05T22:52:52.430 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-03-06T01:33:05.204 INFO:teuthology.results:Tests finished! gathering results... 2022-03-06T01:33:08.599 INFO:scrape:Found 40 jobs 2022-03-06T01:33:09.680 INFO:scrape:Found 4 distinct failure reasons 2022-03-06T01:33:09.681 INFO:scrape:Failure: Cannot create secret 2022-03-06T01:33:09.682 INFO:scrape:1 jobs: ['6722984'] 2022-03-06T01:33:09.683 INFO:scrape:suites: ['1-ceph-install/install', '2-kms/barbican', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability', 'rgw/crypt/{0-cluster/fixed-1', 'supported-random-distro$/{ubuntu_latest}}'] 2022-03-06T01:33:09.684 INFO:scrape: 2022-03-06T01:33:09.684 INFO:scrape:Failure: rgw multisite test failures 2022-03-06T01:33:09.685 INFO:scrape:1 jobs: ['6722988'] 2022-03-06T01:33:09.685 INFO:scrape:suites: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'rgw/multisite/{clusters', 'supported-random-distro$/{centos_8}', 'tasks/test_multi}'] 2022-03-06T01:33:09.686 INFO:scrape: 2022-03-06T01:33:09.686 INFO:scrape:Failure: Command failed on smithi026 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test_account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|.*test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_services_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*test_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticweb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.*test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds|.*test_object_services.ObjectTest.test_create_object_with_transfer_encoding'" 2022-03-06T01:33:09.687 INFO:scrape:1 jobs: ['6722992'] 2022-03-06T01:33:09.687 INFO:scrape:suites: ['frontend/beast', 'overrides', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}'] 2022-03-06T01:33:09.688 INFO:scrape: 2022-03-06T01:33:09.689 INFO:scrape:Failure: Command failed (s3 tests against rgw) on smithi023 with status 1: 'S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a lifecycle' 2022-03-06T01:33:09.689 INFO:scrape:1 jobs: ['6722986'] 2022-03-06T01:33:09.690 INFO:scrape:suites: ['overrides', 'rgw/lifecycle/{cluster', 'supported-random-distro$/{ubuntu_latest}', 'tasks/rgw_s3tests}'] 2022-03-06T01:33:09.691 INFO:scrape: 2022-03-06T01:33:09.693 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 36 Fail ================================================================= [6722984] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:14:26 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722984/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722984/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fb89cf288f874efe87f27db342dd6c68 Cannot create secret [6722986] rgw/lifecycle/{cluster overrides supported-random-distro$/{ubuntu_latest} tasks/rgw_s3tests} ----------------------------------------------------------------- time: 00:15:32 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722986/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722986/ sentry: https://sentry.ceph.com/organizations/ceph/?query=ae0e0e6ed4b547bd950eae8bd7ec0424 Command failed (s3 tests against rgw) on smithi023 with status 1: 'S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a lifecycle' [6722988] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{centos_8} tasks/test_multi} ----------------------------------------------------------------- time: 00:46:13 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722988/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722988/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8d472d69532947cea979f2546027ab15 rgw multisite test failures [6722992] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:20:43 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722992/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722992/ sentry: https://sentry.ceph.com/organizations/ceph/?query=35cc7349b0e1465da3f775ed550e0e1f Command failed on smithi026 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test _account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|. *test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_serv ices_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*tes t_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticw eb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.* test_object_services.PublicObjectTest.test_access_public_container_object_w ithout_using_creds|.*test_object_services.ObjectTest.test_create_object_wit h_transfer_encoding'" Pass ================================================================= [6722985] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} time: 00:14:44 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722985/ [6722987] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload ubuntu_latest} time: 00:08:08 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722987/ [6722989] rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} time: 00:33:54 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722989/ [6722990] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:22:44 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722990/ [6722991] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast supported-random-distro$/{centos_8} tasks/{0-install first ststests}} time: 00:27:58 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722991/ [6722993] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:12:11 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722993/ [6722994] rgw/tools/{centos_latest cluster tasks} time: 02:44:00 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722994/ [6722995] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:28:00 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722995/ [6722996] rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest} time: 00:16:05 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722996/ [6722997] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:10:30 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722997/ [6722998] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} time: 00:27:24 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722998/ [6722999] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_ragweed ubuntu_latest} time: 00:07:07 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6722999/ [6723000] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{centos_8}} time: 00:12:00 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723000/ [6723001] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:20:07 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723001/ [6723002] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 01:46:20 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723002/ [6723003] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} time: 00:13:40 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723003/ [6723004] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_s3tests ubuntu_latest} time: 00:12:53 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723004/ [6723005] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{rhel_8}} time: 00:23:52 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723005/ [6723006] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:10:30 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723006/ [6723007] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{ubuntu_latest}} time: 00:11:17 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723007/ [6723008] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} time: 00:13:28 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723008/ [6723009] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_user_quota ubuntu_latest} time: 00:07:18 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723009/ [6723010] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{rhel_8} tasks/test_multi} time: 00:41:53 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723010/ [6723011] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_latest}} time: 00:09:03 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723011/ [6723012] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:11:29 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723012/ [6723013] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:26:16 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723013/ [6723014] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:11:12 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723014/ [6723015] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} time: 00:13:32 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723015/ [6723016] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_bucket_quota ubuntu_latest} time: 00:08:48 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723016/ [6723017] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_8}} time: 00:11:54 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723017/ [6723018] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:15:01 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723018/ [6723019] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 01:43:46 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723019/ [6723020] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} time: 00:26:12 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723020/ [6723021] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload ubuntu_latest} time: 00:07:57 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723021/ [6723022] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} time: 00:09:19 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723022/ [6723023] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:11:58 info: http://pulpito.ceph.com/yuriw-2022-03-05_16:36:22-rgw-quincy-distro-default-smithi/6723023/