2022-01-21T19:18:06.851 INFO:root:teuthology version: 1.1.0-75f1bcd2 2022-01-21T19:18:06.853 INFO:teuthology.results:subset: '111/120000' 2022-01-21T19:18:06.853 INFO:teuthology.results:seed: '7747' 2022-01-21T19:39:13.249 INFO:root:teuthology version: 1.1.0-75f1bcd2 2022-01-21T19:39:13.278 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-01-22T02:03:41.366 INFO:teuthology.results:Tests finished! gathering results... 2022-01-22T02:03:44.510 INFO:scrape:Found 39 jobs 2022-01-22T02:03:44.652 WARNING:scrape:Not-understood crash indication 2022-01-21T20:01:51.413 INFO:tasks.rgw.c1.client.0.smithi084.stderr:daemon-helper: command crashed with signal 11 2022-01-22T02:03:44.653 WARNING:scrape:Not-understood crash indication 2022-01-22T02:03:44.775 WARNING:scrape:Not-understood crash indication 2022-01-21T20:02:26.951 INFO:tasks.rgw.c1.client.1.smithi022.stderr:daemon-helper: command crashed with signal 11 2022-01-22T02:03:44.776 WARNING:scrape:Not-understood crash indication 2022-01-22T02:03:44.794 WARNING:scrape:Not-understood crash indication 2022-01-21T20:02:26.951 INFO:tasks.rgw.c1.client.1.smithi022.stderr:daemon-helper: command crashed with signal 11 2022-01-22T02:03:44.795 WARNING:scrape:Not-understood crash indication 2022-01-22T02:03:44.812 WARNING:scrape:Not-understood crash indication 2022-01-21T20:02:26.951 INFO:tasks.rgw.c1.client.1.smithi022.stderr:daemon-helper: command crashed with signal 11 2022-01-22T02:03:44.813 WARNING:scrape:Not-understood crash indication 2022-01-22T02:03:45.140 INFO:scrape:Found 3 distinct failure reasons 2022-01-22T02:03:45.141 INFO:scrape:Dead: b'2022-01-22T01:56:26.390 DEBUG:teuthology.exit:Finished running handlers' 2022-01-22T02:03:45.142 INFO:scrape:1 jobs: ['6632348'] 2022-01-22T02:03:45.142 INFO:scrape:suites: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'rgw/multisite/{clusters', 'supported-random-distro$/{rhel_8}', 'tasks/test_multi}'] 2022-01-22T02:03:45.143 INFO:scrape: 2022-01-22T02:03:45.143 INFO:scrape:Failure: rgw multisite test failures 2022-01-22T02:03:45.144 INFO:scrape:1 jobs: ['6632370'] 2022-01-22T02:03:45.144 INFO:scrape:suites: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{centos_8.stream}', 'tasks/test_multi}'] 2022-01-22T02:03:45.144 INFO:scrape: 2022-01-22T02:03:45.146 INFO:scrape:Failure: Command failed on smithi187 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test_account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|.*test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_services_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*test_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticweb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.*test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds|.*test_object_services.ObjectTest.test_create_object_with_transfer_encoding'" 2022-01-22T02:03:45.146 INFO:scrape:1 jobs: ['6632352'] 2022-01-22T02:03:45.147 INFO:scrape:suites: ['frontend/beast', 'overrides', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}'] 2022-01-22T02:03:45.147 INFO:scrape: 2022-01-22T02:03:45.149 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/ failed: 2 dead: 1 running: 0 waiting: 0 queued: 0 passed: 36 Fail ================================================================= [6632352] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:20:11 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632352/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632352/ sentry: https://sentry.ceph.com/organizations/ceph/?query=61cedf79911e4d0fa4c617642d24f0ce Command failed on smithi187 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test _account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|. *test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_serv ices_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*tes t_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticw eb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.* test_object_services.PublicObjectTest.test_access_public_container_object_w ithout_using_creds|.*test_object_services.ObjectTest.test_create_object_wit h_transfer_encoding'" [6632370] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_8.stream} tasks/test_multi} ----------------------------------------------------------------- time: 01:25:48 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632370/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632370/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5369444b2110464f8061f6dec38e1cdc rgw multisite test failures Dead ================================================================= [6632348] rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{rhel_8} tasks/test_multi} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632348/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632348/ hit max job timeout Pass ================================================================= [6632345] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} time: 00:21:46 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632345/ [6632346] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} time: 00:15:11 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632346/ [6632347] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload ubuntu_latest} time: 00:08:38 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632347/ [6632349] rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}} time: 00:32:26 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632349/ [6632350] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_8}} time: 00:13:02 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632350/ [6632351] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast supported-random-distro$/{centos_8.stream} tasks/{0-install first ststests}} time: 00:27:57 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632351/ [6632353] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:12:54 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632353/ [6632354] rgw/tools/{centos_latest cluster tasks} time: 02:44:36 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632354/ [6632355] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:28:08 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632355/ [6632356] rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest} time: 00:16:19 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632356/ [6632357] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:14:15 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632357/ [6632358] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} time: 00:16:06 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632358/ [6632359] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_ragweed ubuntu_latest} time: 00:08:17 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632359/ [6632360] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_latest}} time: 00:09:43 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632360/ [6632361] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:17:27 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632361/ [6632362] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 01:51:42 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632362/ [6632363] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}} time: 00:14:20 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632363/ [6632364] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_s3tests ubuntu_latest} time: 00:13:03 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632364/ [6632365] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_8.stream}} time: 00:12:37 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632365/ [6632366] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:11:42 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632366/ [6632367] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}} time: 00:26:53 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632367/ [6632368] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} time: 00:14:06 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632368/ [6632369] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_user_quota ubuntu_latest} time: 00:08:05 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632369/ [6632371] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_8}} time: 00:13:03 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632371/ [6632372] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota} time: 00:11:43 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632372/ [6632373] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:28:02 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632373/ [6632374] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload} time: 00:11:28 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632374/ [6632375] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{ubuntu_latest}} time: 00:12:07 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632375/ [6632376] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_bucket_quota ubuntu_latest} time: 00:09:01 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632376/ [6632377] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_8}} time: 00:12:45 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632377/ [6632378] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests} time: 00:15:30 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632378/ [6632379] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 01:48:15 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632379/ [6632380] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8.stream}} time: 00:13:44 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632380/ [6632381] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload ubuntu_latest} time: 00:07:54 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632381/ [6632382] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{centos_8}} time: 00:12:54 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632382/ [6632383] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota} time: 00:11:25 info: http://pulpito.ceph.com/yuriw-2022-01-21_15:31:49-rgw-wip-yuri7-testing-2022-01-20-1609-distro-default-smithi/6632383/