2022-09-10T22:47:54.996 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-10T22:47:54.997 INFO:teuthology.results:subset: '111/120000' 2022-09-10T22:47:54.998 INFO:teuthology.results:seed: '7662' 2022-09-10T23:19:15.634 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18 2022-09-10T23:19:15.655 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-09-11T01:51:27.875 INFO:teuthology.results:Tests finished! gathering results... 2022-09-11T01:51:31.311 INFO:scrape:Found 40 jobs 2022-09-11T01:51:32.847 INFO:scrape:Found 5 distinct failure reasons 2022-09-11T01:51:32.848 INFO:scrape:Failure: Command failed on smithi116 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test_account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|.*test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_services_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*test_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticweb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.*test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds|.*test_object_services.ObjectTest.test_create_object_with_transfer_encoding'" 2022-09-11T01:51:32.849 INFO:scrape:1 jobs: ['7025048'] 2022-09-11T01:51:32.850 INFO:scrape:suites: ['frontend/beast', 'overrides', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}'] 2022-09-11T01:51:32.850 INFO:scrape: 2022-09-11T01:51:32.851 INFO:scrape:Failure: Command failed (s3 tests against rgw) on smithi201 with status 1: 'S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a lifecycle' 2022-09-11T01:51:32.852 INFO:scrape:1 jobs: ['7025042'] 2022-09-11T01:51:32.852 INFO:scrape:suites: ['overrides', 'rgw/lifecycle/{cluster', 'supported-random-distro$/{rhel_8}', 'tasks/rgw_s3tests}'] 2022-09-11T01:51:32.853 INFO:scrape: 2022-09-11T01:51:32.853 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f860b69d9ecbf6a250d17b83c1778c894700357d 2022-09-11T01:51:32.854 INFO:scrape:2 jobs: ['7025041', '7025064'] 2022-09-11T01:51:32.854 INFO:scrape:suites intersection: ['overrides', 'rgw/hadoop-s3a/{clusters/fixed-2', 's3a-hadoop}'] 2022-09-11T01:51:32.855 INFO:scrape:suites union: ['hadoop/default', 'hadoop/v32', 'overrides', 'rgw/hadoop-s3a/{clusters/fixed-2', 's3a-hadoop}'] 2022-09-11T01:51:32.855 INFO:scrape: 2022-09-11T01:51:32.856 INFO:scrape:Failure: Cannot create secret 2022-09-11T01:51:32.856 INFO:scrape:1 jobs: ['7025040'] 2022-09-11T01:51:32.857 INFO:scrape:suites: ['1-ceph-install/install', '2-kms/barbican', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability', 'rgw/crypt/{0-cluster/fixed-1', 'supported-random-distro$/{rhel_8}}'] 2022-09-11T01:51:32.857 INFO:scrape: 2022-09-11T01:51:32.858 INFO:scrape:Failure: rgw multisite test failures 2022-09-11T01:51:32.858 INFO:scrape:1 jobs: ['7025044'] 2022-09-11T01:51:32.859 INFO:scrape:suites: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'rgw/multisite/{clusters', 'supported-random-distro$/{rhel_8}', 'tasks/test_multi}'] 2022-09-11T01:51:32.859 INFO:scrape: 2022-09-11T01:51:32.862 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/ failed: 6 dead: 0 running: 0 waiting: 0 queued: 0 passed: 34 Fail ================================================================= [7025040] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} ----------------------------------------------------------------- time: 00:15:03 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025040/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025040/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3b8596c4ec4543bbb8aa2e017c04caab Cannot create secret [7025041] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} ----------------------------------------------------------------- time: 00:04:53 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025041/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025041/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1ee2cc5b12b34e948f0e5a9ac872a2ee Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1 =f860b69d9ecbf6a250d17b83c1778c894700357d [7025042] rgw/lifecycle/{cluster overrides supported-random-distro$/{rhel_8} tasks/rgw_s3tests} ----------------------------------------------------------------- time: 00:26:21 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025042/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025042/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4756d91752fc4d8ba7bedda5c53b0f8a Command failed (s3 tests against rgw) on smithi201 with status 1: 'S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a lifecycle' [7025044] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{rhel_8} tasks/test_multi} ----------------------------------------------------------------- time: 00:56:13 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025044/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025044/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3365e82167d94194b0fc5fc680b14168 rgw multisite test failures [7025048] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:19:08 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025048/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025048/ sentry: https://sentry.ceph.com/organizations/ceph/?query=44972a4d96894937bc0e11891127f86a Command failed on smithi116 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test _account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|. *test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_serv ices_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*tes t_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticw eb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.* test_object_services.PublicObjectTest.test_access_public_container_object_w ithout_using_creds|.*test_object_services.ObjectTest.test_create_object_wit h_transfer_encoding'" [7025064] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} ----------------------------------------------------------------- time: 00:04:38 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025064/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025064/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a626819bd9394520ae0769041ed12a34 Failed to fetch package version from https://shaman.ceph.com/api/search/?st atus=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1 =f860b69d9ecbf6a250d17b83c1778c894700357d Pass ================================================================= [7025043] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload ubuntu_latest} time: 00:07:27 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025043/ [7025045] rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} time: 00:27:47 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025045/ [7025046] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:11:41 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025046/ [7025047] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast supported-random-distro$/{rhel_8} tasks/{0-install first ststests}} time: 00:28:17 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025047/ [7025049] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:11:47 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025049/ [7025050] rgw/tools/{centos_latest cluster tasks} time: 02:48:59 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025050/ [7025051] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:25:49 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025051/ [7025052] rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest} time: 00:15:44 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025052/ [7025053] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:09:51 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025053/ [7025054] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} time: 00:12:17 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025054/ [7025055] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_ragweed ubuntu_latest} time: 00:06:42 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025055/ [7025056] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{rhel_8}} time: 00:11:30 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025056/ [7025057] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:16:26 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025057/ [7025058] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 00:55:14 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025058/ [7025059] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} time: 00:13:12 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025059/ [7025060] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_s3tests ubuntu_latest} time: 00:12:11 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025060/ [7025061] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} time: 00:08:46 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025061/ [7025062] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:10:18 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025062/ [7025063] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{ubuntu_latest}} time: 00:10:46 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025063/ [7025065] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_user_quota ubuntu_latest} time: 00:06:48 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025065/ [7025066] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_8} tasks/test_multi} time: 00:32:22 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025066/ [7025067] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:11:24 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025067/ [7025068] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:11:46 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025068/ [7025069] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:25:44 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025069/ [7025070] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:11:10 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025070/ [7025071] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{ubuntu_latest}} time: 00:10:55 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025071/ [7025072] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_bucket_quota ubuntu_latest} time: 00:08:13 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025072/ [7025073] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_8}} time: 00:10:07 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025073/ [7025074] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:14:50 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025074/ [7025075] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 00:55:59 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025075/ [7025076] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{ubuntu_latest}} time: 00:11:03 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025076/ [7025077] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload ubuntu_latest} time: 00:06:54 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025077/ [7025078] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} time: 00:08:25 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025078/ [7025079] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:10:43 info: http://pulpito.ceph.com/yuriw-2022-09-10_17:34:50-rgw-quincy-release-distro-default-smithi/7025079/