Found 155 jobs Found 6 distinct failure reasons Failure: Test failure: test_acls (tasks.cephfs.test_acls.TestACLs) 1 jobs: ['5729081'] suites: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/acls}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Crash: Command failed (workunit test suites/ffsb.sh) on smithi106 with status 135: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=af9dac1ff568f0ba2132416c425a35d7a342ee96 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' ceph version 16.0.0-8516-gaf9dac1f (af9dac1ff568f0ba2132416c425a35d7a342ee96) pacific (dev) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7f78a07aa980] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x19c) [0x7f78a192d64e] 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x7f78a192d7d8] 6: ceph-mon(+0x7a3892) [0x564b625a3892] 7: (Monitor::~Monitor()+0x9) [0x564b625a38d9] 8: main() 9: __libc_start_main() 10: _start() 1 jobs: ['5728970'] suites: ['2-workunit/ffsb}}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/multifs/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Crash: Command failed on smithi183 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' ceph version 16.0.0-8516-gaf9dac1f (af9dac1ff568f0ba2132416c425a35d7a342ee96) pacific (dev) 1: /lib64/libpthread.so.0(+0x12b20) [0x7f1c91ae7b20] 2: gsignal() 3: abort() 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1b6) [0x560b4e23f5ed] 5: (KernelDevice::_aio_thread()+0x1254) [0x560b4ed68ea4] 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x560b4ed741a1] 7: /lib64/libpthread.so.0(+0x814a) [0x7f1c91add14a] 8: clone() 1 jobs: ['5728960'] suites: ['clusters/1a2s-mds-1c-client-3node', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/workload/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/bluestore-ec-root', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'scrub/no', 'session_timeout', 'tasks/workunit/suites/iogen}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 1 jobs: ['5729025'] suites: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Test failure: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) 1 jobs: ['5728993'] suites: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/damage}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi033 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=af9dac1ff568f0ba2132416c425a35d7a342ee96 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/untar_snap_rm.sh' 1 jobs: ['5728988'] suites: ['2-workunit/cfuse_workunit_snaptests}}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/thrash/multifs/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'session_timeout', 'tasks/{1-thrash/mon', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}']