2021-03-02T23:00:41.457 INFO:root:teuthology version: 1.1.0-3082387b 2021-03-02T23:00:41.458 INFO:teuthology.results:subset: '22/100000' 2021-03-02T23:00:41.459 INFO:teuthology.results:seed: '6756' 2021-03-03T01:11:21.987 INFO:root:teuthology version: 1.1.0-3082387b 2021-03-03T01:11:22.103 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-03T02:25:27.627 INFO:teuthology.results:Tests finished! gathering results... 2021-03-03T02:25:40.604 INFO:scrape:Found 45 jobs 2021-03-03T02:25:42.626 WARNING:scrape:Misunderstood line: Binary file /home/teuthworker/archive/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928034/teuthology.log matches 2021-03-03T02:25:42.627 WARNING:scrape:Misunderstood line: 2021-03-03T02:25:44.074 WARNING:scrape:Misunderstood line: Binary file /home/teuthworker/archive/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927990/teuthology.log matches 2021-03-03T02:25:44.075 WARNING:scrape:Misunderstood line: 2021-03-03T02:25:44.560 INFO:scrape:Found 1 distinct failure reasons 2021-03-03T02:25:44.561 INFO:scrape:Valgrind: 2021-03-03T02:25:44.561 INFO:scrape:4 jobs: ['5928034', '5927990', '5927974', '5927988'] 2021-03-03T02:25:44.562 INFO:scrape:suites intersection: ['overrides'] 2021-03-03T02:25:44.562 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/sts/{centos_latest', 'rgw/tempest/{clusters/fixed-1', 'rgw/verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'ststests', 'tasks/rgw_tempest', 'tasks/{0-install', 'ubuntu_latest}', 'validater/valgrind}', 'webidentity}}'] 2021-03-03T02:25:44.563 INFO:scrape: 2021-03-03T02:25:44.565 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 41 Fail ================================================================= [5927974] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:20:22 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927974/ log: http://qa-proxy.ceph.com/teuthology/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927974/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b33aaee866ff415994a6563ee6a2f7f9 Command failed on smithi104 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test _account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|. *test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_serv ices_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*tes t_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticw eb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.* test_object_services.PublicObjectTest.test_access_public_container_object_w ithout_using_creds'" [5927988] rgw/sts/{centos_latest clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/{0-install ststests webidentity}} ----------------------------------------------------------------- time: 00:26:40 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927988/ log: http://qa-proxy.ceph.com/teuthology/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927988/ "2021-03-03T00:26:13.012103+0000 mon.a (mon.0) 229 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log [5927990] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 01:20:18 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927990/ log: http://qa-proxy.ceph.com/teuthology/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927990/ saw valgrind issues [5928034] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} ----------------------------------------------------------------- time: 01:06:25 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928034/ log: http://qa-proxy.ceph.com/teuthology/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928034/ saw valgrind issues Pass ================================================================= [5927960] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:27:50 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927960/ [5927962] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests}} time: 00:19:27 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927962/ [5927964] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} time: 00:13:54 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927964/ [5927966] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_ragweed} time: 00:08:09 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927966/ [5927968] rgw/multisite/{clusters frontend/beast omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} time: 00:29:50 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927968/ [5927970] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_8}} time: 00:12:29 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927970/ [5927972] rgw/sts/{centos_latest clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/{0-install ststests webidentity}} time: 00:26:29 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927972/ [5927976] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_bucket_quota} time: 00:13:17 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927976/ [5927978] rgw/tools/{centos_latest cluster tasks} time: 00:18:59 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927978/ [5927980] rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest} time: 00:17:08 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927980/ [5927982] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_multipart_upload} time: 00:10:23 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927982/ [5927984] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_s3tests} time: 00:12:10 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927984/ [5927986] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_latest}} time: 00:10:01 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927986/ [5927992] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests}} time: 00:11:57 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927992/ [5927994] rgw/multisite/{clusters frontend/civetweb omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} time: 00:20:51 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927994/ [5927996] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_s3tests} time: 00:17:33 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927996/ [5927998] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_user_quota} time: 00:07:34 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5927998/ [5928000] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_8}} time: 00:12:23 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928000/ [5928002] rgw/sts/{centos_latest clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/{0-install ststests webidentity}} time: 00:27:45 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928002/ [5928004] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:10:47 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928004/ [5928006] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:31:02 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928006/ [5928008] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests}} time: 00:11:55 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928008/ [5928010] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} time: 00:12:50 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928010/ [5928012] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_bucket_quota} time: 00:08:18 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928012/ [5928014] rgw/multisite/{clusters frontend/civetweb omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} time: 00:31:11 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928014/ [5928016] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:14:26 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928016/ [5928018] rgw/sts/{centos_latest clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/{0-install ststests webidentity}} time: 00:25:49 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928018/ [5928020] rgw/tempest/{clusters/fixed-1 frontend/civetweb overrides tasks/rgw_tempest ubuntu_latest} time: 00:19:43 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928020/ [5928022] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_bucket_quota} time: 00:12:11 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928022/ [5928024] rgw/website/{clusters/fixed-2 frontend/civetweb http overrides tasks/s3tests-website ubuntu_latest} time: 00:15:01 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928024/ [5928026] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_multipart_upload} time: 00:10:40 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928026/ [5928028] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_multipart_upload} time: 00:07:10 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928028/ [5928030] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{rhel_8}} time: 00:14:26 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928030/ [5928032] rgw/sts/{centos_latest clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/{0-install ststests webidentity}} time: 00:25:55 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928032/ [5928036] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests}} time: 00:12:10 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928036/ [5928038] rgw/multisite/{clusters frontend/beast omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} time: 00:20:39 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928038/ [5928040] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_s3tests} time: 00:16:08 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928040/ [5928042] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_ragweed} time: 00:06:43 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928042/ [5928044] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{rhel_8}} time: 00:14:27 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928044/ [5928046] rgw/sts/{centos_latest clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/{0-install ststests webidentity}} time: 00:25:22 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928046/ [5928048] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:11:36 info: http://pulpito.ceph.com/hmunjulu-2021-03-02_21:00:43-rgw-wip-hmunjulu01-testing-2021-03-02-1039-distro-basic-smithi/5928048/