2023-09-21T01:00:08.276 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-21T01:00:08.277 INFO:teuthology.results:subset: '1/2' 2023-09-21T01:00:08.278 INFO:teuthology.results:seed: '9340' 2023-09-21T01:09:23.460 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-21T01:09:23.478 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-09-21T04:31:38.830 INFO:teuthology.results:Tests finished! gathering results... 2023-09-21T04:31:39.752 INFO:scrape:Found 12 jobs 2023-09-21T04:31:40.510 INFO:scrape:Found 3 distinct failure reasons 2023-09-21T04:31:40.511 INFO:scrape:Crash: Command failed (workunit test rgw/test_librgw_file.sh) on smithi160 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=adcb54a3689e4b3aad81de4dbb06f5f1de3a1e11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_librgw_file.sh' 2023-09-21T04:31:40.512 INFO:scrape:ceph version 18.0.0-6155-gadcb54a3 (adcb54a3689e4b3aad81de4dbb06f5f1de3a1e11) reef (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7faf80054db0] 2: /lib64/libc.so.6(+0xa154c) [0x7faf800a154c] 3: raise() 4: abort() 5: /lib64/libtcmalloc.so.4(+0x161f4) [0x7faf7de161f4] 6: /lib64/libtcmalloc.so.4(+0x1a7e3) [0x7faf7de1a7e3] 7: (RGWSystemMetaObj::read_id(DoutPrefixProvider const*, std::__cxx11::basic_string, std::allocator > const&, std::__cxx11::basic_string, std::allocator >&, optional_yield)+0x294) [0x7faf58ba1534] 8: (RGWZoneParams::read_default_id(DoutPrefixProvider const*, std::__cxx11::basic_string, std::allocator >&, optional_yield, bool)+0x17f) [0x7faf58b9326f] 9: (RGWSystemMetaObj::init(DoutPrefixProvider const*, ceph::common::CephContext*, RGWSI_SysObj*, optional_yield, bool, bool)+0x15d) [0x7faf58b92cfd] 10: (RGWSI_Zone::do_start(optional_yield, DoutPrefixProvider const*)+0x47a) [0x7faf58be7bfa] 11: (RGWServiceInstance::start(optional_yield, DoutPrefixProvider const*)+0x20) [0x7faf58cc69a0] 12: (RGWSI_Notify::do_start(optional_yield, DoutPrefixProvider const*)+0x3c) [0x7faf58d6524c] 13: (RGWServiceInstance::start(optional_yield, DoutPrefixProvider const*)+0x20) [0x7faf58cc69a0] 14: (RGWServices_Def::init(ceph::common::CephContext*, bool, bool, bool, optional_yield, DoutPrefixProvider const*)+0xd27) [0x7faf58ccc157] 15: (RGWServices::do_init(ceph::common::CephContext*, bool, bool, bool, optional_yield, DoutPrefixProvider const*)+0x2c) [0x7faf58ccdd4c] 16: (RGWRados::init_svc(bool, DoutPrefixProvider const*)+0x5b) [0x7faf58c0f24b] 17: (RGWRados::init_begin(DoutPrefixProvider const*)+0x32) [0x7faf58c153f2] 18: new_Driver() 19: (DriverManager::init_storage_provider(DoutPrefixProvider const*, ceph::common::CephContext*, DriverManager::Config const&, bool, bool, bool, bool, bool, bool, bool, bool, optional_yield)+0x183) [0x7faf81af00c3] 20: (rgw::AppMain::init_storage()+0x280) [0x7faf817a1480] 21: (rgw::RGWLib::init(std::vector >&)+0x3ea) [0x7faf8188319a] 22: librgw_create() 23: (LibRGW_INIT_Test::TestBody()+0x43) [0x55861ca1d7c3] 24: ceph_test_librgw_file_nfsns(+0x6bfec) [0x55861ca5cfec] 25: (testing::Test::Run()+0xe3) [0x55861ca41323] 26: ceph_test_librgw_file_nfsns(+0x6c1c5) [0x55861ca5d1c5] 27: ceph_test_librgw_file_nfsns(+0x6c359) [0x55861ca5d359] 28: (testing::internal::UnitTestImpl::RunAllTests()+0x5d6) [0x55861ca4d9f6] 29: (testing::UnitTest::Run()+0x1a8) [0x55861ca4c048] 30: main() 31: /lib64/libc.so.6(+0x3feb0) [0x7faf8003feb0] 2023-09-21T04:31:40.512 INFO:scrape:1 jobs: ['7399939'] 2023-09-21T04:31:40.513 INFO:scrape:suites: ['clusters/fixed-2', 'datacache/no_datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw:verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{cls', 'validater/valgrind}', 'versioning}'] 2023-09-21T04:31:40.513 INFO:scrape: 2023-09-21T04:31:40.514 INFO:scrape:Crash: Command failed (workunit test rgw/test_librgw_file.sh) on smithi012 with status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=adcb54a3689e4b3aad81de4dbb06f5f1de3a1e11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_librgw_file.sh' 2023-09-21T04:31:40.514 INFO:scrape:ceph version 18.0.0-6155-gadcb54a3 (adcb54a3689e4b3aad81de4dbb06f5f1de3a1e11) reef (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7f9956454db0] 2: /lib64/libc.so.6(+0xa154c) [0x7f99564a154c] 3: raise() 4: abort() 5: /lib64/libc.so.6(+0x29130) [0x7f9956429130] 6: /lib64/libc.so.6(+0xab617) [0x7f99564ab617] 7: /lib64/libc.so.6(+0xab8ac) [0x7f99564ab8ac] 8: free() 9: /usr/lib64/ceph/libceph-common.so.2(+0x1f07f0) [0x7f9956df07f0] 10: (ProtocolV2::handle_auth_signature(ceph::buffer::v15_2_0::list&)+0x3e4) [0x7f9957029b64] 11: (ProtocolV2::run_continuation(Ct&)+0x39) [0x7f995700aba9] 12: (AsyncConnection::process()+0x686) [0x7f9956fe9116] 13: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0x1cd) [0x7f9957031f4d] 14: /usr/lib64/ceph/libceph-common.so.2(+0x432a56) [0x7f9957032a56] 15: /lib64/libstdc++.so.6(+0xdb924) [0x7f99568db924] 16: /lib64/libc.so.6(+0x9f802) [0x7f995649f802] 17: /lib64/libc.so.6(+0x3f450) [0x7f995643f450] 2023-09-21T04:31:40.514 INFO:scrape:1 jobs: ['7399938'] 2023-09-21T04:31:40.515 INFO:scrape:suites: ['clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{cls', 'validater/lockdep}', 'versioning}'] 2023-09-21T04:31:40.515 INFO:scrape: 2023-09-21T04:31:40.516 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rgw/test_librgw_file.sh 2023-09-21T04:31:40.516 INFO:scrape:1 jobs: ['7399932'] 2023-09-21T04:31:40.517 INFO:scrape:suites: ['clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw:verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'supported-random-distro$/{centos_latest}', 'tasks/{cls', 'validater/lockdep}', 'versioning}'] 2023-09-21T04:31:40.517 INFO:scrape: