Found 216 jobs Found 8 distinct failure reasons Failure: '2021-08-03T11:28:55.954859+00:00 smithi119 kernel: WARNING: CPU: 0 PID: 42319 at fs/ceph/mds_client.c:4495 check_session_state+0x55/0x60 [ceph] ' in syslog 94 jobs suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'osd}', 'overrides/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}'] suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8.stream}', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'rhel_8}', 'tasks/acls}', 'tasks/admin}', 'tasks/alternate-pool}', 'tasks/auto-repair}', 'tasks/backtrace}', 'tasks/cap-flush}', 'tasks/client-readahead}', 'tasks/client-recovery}', 'tasks/damage}', 'tasks/exports}', 'tasks/forward-scrub}', 'tasks/fragment}', 'tasks/journal-repair}', 'tasks/mds-flush}', 'tasks/multimds_misc}', 'tasks/openfiletable}', 'tasks/pool-perm}', 'tasks/scrub}', 'tasks/sessionmap}', 'tasks/snap-schedule}', 'tasks/strays}', 'tasks/test_journal_migration}', 'tasks/volumes/{overrides', 'test/basic}}', 'test/clone}}', 'test/misc}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Test failure: test_ephemeral_random_failover (tasks.cephfs.test_exports.TestEphemeralPins) 1 jobs: ['6309140'] suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8.stream}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/exports}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Test failure: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 1 jobs: ['6309190'] suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/snap-schedule}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: No module named 'tasks.ceph' 8 jobs: ['6309202', '6309185', '6309186', '6309187', '6309200', '6309201', '6309205', '6309204'] suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin', 'mds', 'mon', 'no_client_pidfile', 'osd}', 'overrides/{frag_enable', 'whitelist_health', 'whitelist_wrongly_marked_down}'] suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/fuse', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/sessionmap}', 'tasks/truncate_delay}', 'tasks/workunit/quota}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Command failed on smithi173 with status 1: 'sudo yum -y install ceph-radosgw ceph-test ceph ceph-fuse cephfs-java libcephfs_jni1 libcephfs1 librados2 librbd1 python-ceph rbd-fuse python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel' 3 jobs: ['6309199', '6309198', '6309197'] suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/test_journal_migration}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/test_journal_migration}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Test failure: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 6 jobs: ['6309160', '6309050', '6309051', '6309159', '6309052', '6309158'] suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/mds-full}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Crash: '2021-08-03T10:57:20.542653+00:00 smithi144 kernel: WARNING: CPU: 7 PID: 39112 at fs/ceph/mds_client.c:4456 check_session_state+0x4e/0x60 [ceph] ' in syslog ceph version 17.0.0-6107-ge1ccf367 (e1ccf367dff0b9d57bd5c50477cfa60c76c64762) quincy (dev) 1: /lib64/libpthread.so.0(+0x12b20) [0x7fdcaffa9b20] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x7fdcb1209048] 5: /usr/lib64/ceph/libceph-common.so.2(+0x28220b) [0x7fdcb120920b] 6: (MDSTableClient::handle_request(boost::intrusive_ptr const&)+0x128b) [0x56264235f8eb] 7: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe1c) [0x5626420a23ac] 8: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x7bb) [0x5626420a47bb] 9: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x55) [0x5626420a4db5] 10: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x128) [0x562642094b58] 11: (DispatchQueue::entry()+0x14ca) [0x7fdcb147859a] 12: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fdcb1526ba1] 13: (Thread::_entry_func(void*)+0xd) [0x7fdcb1302bdd] 14: /lib64/libpthread.so.0(+0x814a) [0x7fdcaff9f14a] 15: clone() 3 jobs: ['6309084', '6309083', '6309085'] suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/snapshots}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/snapshots}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] Failure: Test failure: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 1 jobs: ['6309188'] suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/snap-schedule}', 'whitelist_health', 'whitelist_wrongly_marked_down}']