Found 34 jobs Missing teuthology log /home/teuthworker/archive/matan-2024-01-17_15:47:21-crimson-rados-wip-matanb-crimson-testing-17.1-crimson-only-distro-crimson-smithi/7518871/teuthology.log Found 3 distinct failure reasons Crash: Command failed on smithi086 with status 6: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats' ceph version 19.0.0-720-gcecb6cb5 (cecb6cb5cdba1d67db289980b0e46cfe0334a30f) squid (dev) 1: /lib64/libc.so.6(+0x54db0) [0x7f1680454db0] 2: /lib64/libc.so.6(+0xa154c) [0x7f16804a154c] 3: raise() 4: abort() 5: /lib64/libstdc++.so.6(+0xa1a01) [0x7f16808a1a01] 6: /lib64/libstdc++.so.6(+0xad37c) [0x7f16808ad37c] 7: /lib64/libstdc++.so.6(+0xad3e7) [0x7f16808ad3e7] 8: /lib64/libstdc++.so.6(+0xad649) [0x7f16808ad649] 9: (ceph::buffer::v15_2_0::list::iterator_impl::copy(unsigned int, ceph::buffer::v15_2_0::list&)+0x5a) [0x7f16810b4be0] 10: /usr/lib64/ceph/libceph-common.so.2(+0x17380b) [0x7f1680d7380b] 11: (OSDMap::Incremental::decode(ceph::buffer::v15_2_0::list::iterator_impl&)+0x510) [0x7f16811964e2] 12: /lib64/librados.so.2(+0xede80) [0x7f1681afbe80] 13: /lib64/librados.so.2(+0x101665) [0x7f1681b0f665] 14: /lib64/librados.so.2(+0xf9f70) [0x7f1681b07f70] 15: /lib64/librados.so.2(+0xb773e) [0x7f1681ac573e] 16: /usr/lib64/ceph/libceph-common.so.2(+0x362c59) [0x7f1680f62c59] 17: (DispatchQueue::entry()+0x833) [0x7f1680f637bd] 18: /usr/lib64/ceph/libceph-common.so.2(+0x43b69f) [0x7f168103b69f] 19: (Thread::entry_wrapper()+0x43) [0x7f1680dff207] 20: (Thread::_entry_func(void*)+0xd) [0x7f1680dff223] 21: /lib64/libc.so.6(+0x9f802) [0x7f168049f802] 22: /lib64/libc.so.6(+0x3f450) [0x7f168043f450] 1 jobs: ['7518857'] suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'clusters/{fixed-2}', 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'objectstore/bluestore', 'thrashers/default', 'thrashosds-health', 'workloads/small-objects-balanced}'] Dead: None 1 jobs: ['7518871'] Timeout 3h running clone.client.0/qa/workunits/rados/test.sh --crimson 1 jobs: ['7518842'] suites: ['crimson-rados/basic/{clusters/fixed-2', 'crimson-supported-all-distro/centos_latest', 'crimson_qa_overrides', 'deploy/ceph', 'tasks/rados_api_tests}']