Found 155 jobs Found 10 distinct failure reasons Failure: Test failure: test_evict_client (tasks.cephfs.test_volume_client.TestVolumeClient) 1 jobs: ['5738903'] suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs/volumes/{begin', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/volume-client}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Test failure: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 1 jobs: ['5738896'] suites: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/snap-schedule}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Test failure: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) 1 jobs: ['5738971'] suites: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/damage}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Test failure: test_idempotency (tasks.cephfs.test_volume_client.TestVolumeClient) 1 jobs: ['5739024'] suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/volumes/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/volume-client}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Test failure: test_acls (tasks.cephfs.test_acls.TestACLs) 1 jobs: ['5738923'] suites: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/acls}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: "2020-12-27T07:29:54.531808+0000 mds.c (mds.0) 149 : cluster [WRN] Scrub error on inode 0x1000000d84b (/client.0/tmp/payload.1/multiple_rsync_payload.124740/python2.7/dist-packages/samba/tests) see mds.c log and `damage ls` output for details" in cluster log 1 jobs: ['5738989'] suites: ['clusters/1a2s-mds-1c-client-3node', 'conf/{client', 'distro/{rhel_8}', 'fs/workload/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore-ec/filestore-xfs', 'omap_limit/10000', 'osd-asserts', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'scrub/yes', 'session_timeout', 'tasks/workunit/fs/misc}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Command failed (workunit test libcephfs/test.sh) on smithi087 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3ef56c398ba628e9011753b67786b8e2a37447a6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/libcephfs/test.sh' 1 jobs: ['5738906'] suites: ['clusters/1-mds-1-client-coloc', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/libcephfs/{begin', 'mds', 'mon', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/libcephfs}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Crash: Command failed (workunit test suites/fsstress.sh) on smithi045 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3ef56c398ba628e9011753b67786b8e2a37447a6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh' ceph version 16.0.0-8592-g3ef56c39 (3ef56c398ba628e9011753b67786b8e2a37447a6) pacific (dev) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7f15fcbd1980] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x19c) [0x7f15fdd523ce] 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x7f15fdd52558] 6: ceph-mon(+0x7a1d02) [0x55d837ba1d02] 7: (Monitor::~Monitor()+0x9) [0x55d837ba1d49] 8: main() 9: __libc_start_main() 10: _start() 1 jobs: ['5738977'] suites: ['2-workunit/cfuse_workunit_suites_fsstress}}', 'clusters/1a3s-mds-2c-client', 'conf/{client', 'distro/{centos_8}', 'fs/thrash/multifs/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multifs', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'session_timeout', 'tasks/{1-thrash/mds', 'thrashosds-health', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Command failed (workunit test fs/quota/quota.sh) on smithi060 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3ef56c398ba628e9011753b67786b8e2a37447a6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/quota/quota.sh' 1 jobs: ['5738919'] suites: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'distro/{rhel_8}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/workunit/quota}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 1 jobs: ['5739004'] suites: ['clusters/1-mds-4-client-coloc', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}']