2022-09-14T09:38:07.816 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18
2022-09-14T09:38:07.818 INFO:teuthology.results:subset: '1/2'
2022-09-14T09:38:07.818 INFO:teuthology.results:seed: '9178'
2022-09-14T12:48:50.640 INFO:root:teuthology version: 1.1.1.dev418+gb58f2c18
2022-09-14T12:48:50.683 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-09-14T23:46:45.753 INFO:teuthology.results:Tests finished! gathering results...
2022-09-14T23:46:52.598 INFO:scrape:Found 71 jobs
2022-09-14T23:46:52.955 WARNING:scrape:Not-understood crash indication 2022-09-14T12:16:50.778 INFO:tasks.rgw.c2.client.0.smithi046.stderr:daemon-helper: command crashed with signal 11
2022-09-14T23:46:52.957 WARNING:scrape:Not-understood crash indication 2022-09-14T12:16:53.798 INFO:tasks.rgw.c2.client.1.smithi046.stderr:daemon-helper: command crashed with signal 11
2022-09-14T23:46:52.957 WARNING:scrape:Not-understood crash indication 2022-09-14T14:02:04.460 INFO:tasks.rgw.c1.client.0.smithi005.stderr:daemon-helper: command crashed with signal 11
2022-09-14T23:46:52.958 WARNING:scrape:Not-understood crash indication 
2022-09-14T23:46:55.340 INFO:scrape:Found 6 distinct failure reasons
2022-09-14T23:46:55.343 INFO:scrape:Dead: b'2022-09-14T23:42:16.938 DEBUG:teuthology.exit:Finished running handlers'
2022-09-14T23:46:55.343 INFO:scrape:1 jobs: ['7030987']
2022-09-14T23:46:55.344 INFO:scrape:suites: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'rgw/multisite/{clusters', 'supported-random-distro$/{centos_8}', 'tasks/test_multi}']
2022-09-14T23:46:55.345 INFO:scrape:
2022-09-14T23:46:55.346 INFO:scrape:Failure: Cannot create secret
2022-09-14T23:46:55.347 INFO:scrape:1 jobs: ['7030982']
2022-09-14T23:46:55.348 INFO:scrape:suites: ['1-ceph-install/install', '2-kms/barbican', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability', 'rgw/crypt/{0-cluster/fixed-1', 's3tests-branch', 'supported-random-distro$/{centos_8}}']
2022-09-14T23:46:55.348 INFO:scrape:
2022-09-14T23:46:55.349 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=6f2650a3527c5e97723b3bc5d52d9e9ac2bb6c18
2022-09-14T23:46:55.350 INFO:scrape:2 jobs: ['7031022', '7030984']
2022-09-14T23:46:55.351 INFO:scrape:suites intersection: ['overrides', 'rgw/hadoop-s3a/{clusters/fixed-2', 's3a-hadoop}']
2022-09-14T23:46:55.352 INFO:scrape:suites union: ['hadoop/default', 'hadoop/v32', 'overrides', 'rgw/hadoop-s3a/{clusters/fixed-2', 's3a-hadoop}']
2022-09-14T23:46:55.353 INFO:scrape:
2022-09-14T23:46:55.354 INFO:scrape:Failure: [Errno 2] No such file or directory: '/home/teuthworker/src/git.ceph.com_ceph-c_6f2650a3527c5e97723b3bc5d52d9e9ac2bb6c18/qa/tasks/boto.cfg.template'
2022-09-14T23:46:55.355 INFO:scrape:1 jobs: ['7030990']
2022-09-14T23:46:55.356 INFO:scrape:suites: ['first', 'ignore-pg-availability', 'objectstore', 'overrides', 'pool-type', 'rgw/sts/{cluster', 'rgw_frontend/beast', 's3tests-branch', 'ststests}}', 'supported-random-distro$/{rhel_8}', 'tasks/{0-install']
2022-09-14T23:46:55.357 INFO:scrape:
2022-09-14T23:46:55.358 INFO:scrape:Crash: rgw multisite test failures
2022-09-14T23:46:55.361 INFO:scrape:ceph version 17.0.0-14918-g6f2650a3 (6f2650a3527c5e97723b3bc5d52d9e9ac2bb6c18) quincy (dev)
 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f0c9f8cccf0]
 2: /lib64/libc.so.6(+0xc96e0) [0x7f0c9cf856e0]
 3: (std::_Rb_tree<rgw_obj_key, std::pair<rgw_obj_key const, RGWSyncShardMarkerTrack<rgw_obj_key, rgw_obj_key>::marker_entry>, std::_Select1st<std::pair<rgw_obj_key const, RGWSyncShardMarkerTrack<rgw_obj_key, rgw_obj_key>::marker_entry> >, std::less<rgw_obj_key>, std::allocator<std::pair<rgw_obj_key const, RGWSyncShardMarkerTrack<rgw_obj_key, rgw_obj_key>::marker_entry> > >::find(rgw_obj_key const&)+0x6f) [0x5598486bc6af]
 4: (RGWBucketSyncSingleEntryCR<rgw_obj_key, rgw_obj_key>::operate(DoutPrefixProvider const*)+0x8bc) [0x5598486e7f9c]
 5: (RGWCoroutinesStack::operate(DoutPrefixProvider const*, RGWCoroutinesEnv*)+0x18f) [0x5598487e18ef]
 6: (RGWCoroutinesManager::run(DoutPrefixProvider const*, std::__cxx11::list<RGWCoroutinesStack*, std::allocator<RGWCoroutinesStack*> >&)+0x521) [0x5598487e20c1]
 7: (RGWBucketPipeSyncStatusManager::run(DoutPrefixProvider const*)+0xaca) [0x55984869251a]
 8: main()
 9: __libc_start_main()
 10: _start()
2022-09-14T23:46:55.361 INFO:scrape:1 jobs: ['7031023']
2022-09-14T23:46:55.363 INFO:scrape:suites: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{centos_8}', 'tasks/test_multi}']
2022-09-14T23:46:55.364 INFO:scrape:
2022-09-14T23:46:55.365 INFO:scrape:Failure: Command failed on smithi203 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test_account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|.*test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_services_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*test_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticweb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.*test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds|.*test_object_services.ObjectTest.test_create_object_with_transfer_encoding'"
2022-09-14T23:46:55.366 INFO:scrape:1 jobs: ['7030991']
2022-09-14T23:46:55.367 INFO:scrape:suites: ['frontend/beast', 'overrides', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}']
2022-09-14T23:46:55.368 INFO:scrape: