Found 340 jobs
Found 7 distinct failure reasons
Failure: "2020-10-09T22:25:53.661640+0000 mds.a (mds.7) 1 : cluster [WRN] client.4648 isn't responding to mclientcaps(revoke), ino 0x2000000026e pending pAsLsXsFscr issued pAsLsXsFsxcrwb, sent 64.110332 seconds ago" in cluster log
2 jobs: ['5508843', '5508911']
suites intersection: ['begin', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/yes', 'mds', 'mon', 'mount/fuse', 'osd}', 'overrides/{basic/{frag_enable', 'q_check_counter/check_counter', 'tasks/cfuse_workunit_suites_ffsb}', 'whitelist_health', 'whitelist_wrongly_marked_down}']
suites union: ['begin', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/yes', 'mds', 'mon', 'mount/fuse', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest}', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{basic/{frag_enable', 'q_check_counter/check_counter', 'tasks/cfuse_workunit_suites_ffsb}', 'whitelist_health', 'whitelist_wrongly_marked_down}']

Failure: 
4 jobs: ['5508850', '5508884', '5508918', '5508986']
suites intersection: ['centos_latest', 'clusters/3-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'multimds/verify/{begin', 'osd}', 'overrides/{fuse-default-perm-no', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']
suites union: ['centos_latest', 'clusters/3-mds', 'conf/{client', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'multimds/verify/{begin', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{fuse-default-perm-no', 'tasks/cfuse_workunit_suites_fsstress', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}}']

Failure: Command failed on smithi063 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel'
5 jobs: ['5508867', '5508841', '5508952', '5508995', '5508908']
suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'whitelist_health']
suites union: ['begin', 'centos_latest', 'ceph-thrash/mon', 'clusters/3-mds', 'clusters/9-mds', 'clusters/9-mds-3-standby', 'conf/{client', 'fuse-default-perm-no}', 'inline/no', 'inline/yes', 'k-testing}', 'mds', 'mon', 'mon-debug', 'mount', 'mount/fuse', 'mount/kclient/{k-testing', 'mount/kclient/{mount', 'ms-die-on-skipped}', 'ms-die-on-skipped}}', 'msgr-failures/none', 'multimds/basic/{0-supported-random-distro$/{centos_8}', 'multimds/basic/{0-supported-random-distro$/{rhel_8}', 'multimds/thrash/{0-supported-random-distro$/{ubuntu_latest}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-bitmap', 'objectstore-ec/bluestore-comp', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{fuse-default-perm-no', 'q_check_counter/check_counter', 'tasks/cfuse_workunit_suites_blogbench}', 'tasks/cfuse_workunit_suites_dbench', 'tasks/cfuse_workunit_suites_dbench}', 'tasks/cfuse_workunit_suites_fsstress', 'tasks/cfuse_workunit_suites_fsstress}', 'thrash/{frag_enable', 'thrash_debug}', 'validater/lockdep}', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'whitelist_wrongly_marked_down}}']

Failure: "2020-10-10T01:20:51.319145+0000 mds.g (mds.4) 1 : cluster [WRN] client.4526 isn't responding to mclientcaps(revoke), ino 0x500000007e8 pending pAsLsXsFscr issued pAsLsXsFscr, sent 63.020861 seconds ago" in cluster log
1 jobs: ['5509078']
suites: ['begin', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/yes', 'mds', 'mon', 'mount/fuse', 'multimds/basic/{0-supported-random-distro$/{ubuntu_latest}', 'objectstore-ec/filestore-xfs', 'osd}', 'overrides/{basic/{frag_enable', 'q_check_counter/check_counter', 'tasks/cfuse_workunit_suites_blogbench}', 'whitelist_health', 'whitelist_wrongly_marked_down}']

Failure: Command failed (workunit test suites/fsx.sh) on smithi175 with status 161: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57ce856eed04586d512872a4560df2552f7c05f3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsx.sh'
1 jobs: ['5508914']
suites: ['begin', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/yes', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{rhel_8}', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/testing/{flavor/centos_latest', 'q_check_counter/check_counter', 'tasks/cfuse_workunit_suites_fsx}', 'whitelist_health', 'whitelist_wrongly_marked_down}']

Dead: b'2020-10-10T12:03:12.313 DEBUG:teuthology.exit:Finished running handlers'
3 jobs: ['5509020', '5508857', '5508855']
suites intersection: ['conf/{client', 'mds', 'mon', 'osd}', 'whitelist_health']
suites union: ['begin', 'centos_latest', 'clusters/3-mds', 'clusters/9-mds', 'conf/{client', 'fuse-default-perm-no}', 'inline/yes', 'k-testing}', 'mds', 'mon', 'mon-debug', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'multimds/basic/{0-supported-random-distro$/{rhel_8}', 'multimds/verify/{begin', 'objectstore-ec/bluestore-comp-ec-root', 'objectstore-ec/bluestore-ec-root', 'osd}', 'overrides/{basic/{frag_enable', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{fuse-default-perm-no', 'q_check_counter/check_counter', 'tasks/cfuse_workunit_misc}', 'tasks/cfuse_workunit_suites_blogbench}', 'tasks/cfuse_workunit_suites_dbench', 'validater/valgrind}', 'verify/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}', 'whitelist_wrongly_marked_down}}']

Failure: "2020-10-09T23:38:05.951007+0000 mon.b (mon.0) 166 : cluster [WRN] Health check failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster log
1 jobs: ['5508964']
suites: ['begin', 'ceph-thrash/mds', 'clusters/3-mds-2-standby', 'conf/{client', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'msgr-failures/osd-mds-delay', 'multimds/thrash/{0-supported-random-distro$/{rhel_8}', 'objectstore-ec/bluestore-comp-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{fuse-default-perm-no', 'tasks/cfuse_workunit_suites_fsstress}', 'thrash/{frag_enable', 'thrash_debug}', 'whitelist_health', 'whitelist_wrongly_marked_down}']