2022-06-03T16:59:48.822 INFO:root:teuthology version: 1.1.1.dev360+g53fc7fef 2022-06-03T16:59:48.823 INFO:teuthology.results:seed: '6505' 2022-06-03T17:03:48.710 INFO:root:teuthology version: 1.1.1.dev360+g53fc7fef 2022-06-03T17:03:48.778 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-06-03T18:24:55.134 INFO:teuthology.results:Tests finished! gathering results... 2022-06-03T18:24:55.511 INFO:scrape:Found 6 jobs 2022-06-03T18:24:56.189 INFO:scrape:Found 1 distinct failure reasons 2022-06-03T18:24:56.189 INFO:scrape:Crash: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining), test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2022-06-03T18:24:56.190 INFO:scrape:ceph version 17.0.0-12513-gbed2db37 (bed2db37782ac79696a1b2531149adadbbbf8514) quincy (dev) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x143c0) [0x7f5ec4f213c0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b7) [0x7f5ec447d834] 5: /usr/lib/ceph/libceph-common.so.2(+0x27d9de) [0x7f5ec447d9de] 6: (ScrubStack::_enqueue(MDSCacheObject*, std::shared_ptr&, bool)+0x951) [0x5637d9fc8851] 7: (ScrubStack::handle_scrub(boost::intrusive_ptr const&)+0x1095) [0x5637d9fd1fc5] 8: (ScrubStack::dispatch(boost::intrusive_ptr const&)+0x1a4) [0x5637d9fd3164] 9: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x1cd) [0x5637d9c90bfd] 10: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x643) [0x5637d9c93f53] 11: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5637d9c945cc] 12: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1d6) [0x5637d9c823d6] 13: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x460) [0x7f5ec4719790] 14: (DispatchQueue::entry()+0x58f) [0x7f5ec471702f] 15: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5ec47e7c31] 16: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f5ec4f15609] 17: clone() 2022-06-03T18:24:56.190 INFO:scrape:2 jobs: ['6862768', '6862771'] 2022-06-03T18:24:56.190 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/uninlining}'] 2022-06-03T18:24:56.190 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/uninlining}'] 2022-06-03T18:24:56.190 INFO:scrape: 2022-06-03T18:24:56.191 INFO:teuthology.results:Sending results to mchangir@redhat.com: Test Run: mchangir-2022-06-03_16:57:52-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/mchangir-2022-06-03_16:57:52-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/mchangir-2022-06-03_16:57:52-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 4 Fail ================================================================= [6862768] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/uninlining} ----------------------------------------------------------------- time: 00:25:22 info: http://pulpito.ceph.com/mchangir-2022-06-03_16:57:52-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6862768/ log: http://qa-proxy.ceph.com/teuthology/mchangir-2022-06-03_16:57:52-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6862768/ sentry: https://sentry.ceph.com/organizations/ceph/?query=cbbd656f578849afbb2d4bbc7c1bace6 Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining), test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) [6862771] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/uninlining} ----------------------------------------------------------------- time: 00:32:26 info: http://pulpito.ceph.com/mchangir-2022-06-03_16:57:52-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6862771/ log: http://qa-proxy.ceph.com/teuthology/mchangir-2022-06-03_16:57:52-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6862771/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0bce0f61d0df4a9285b6dfa17172a0fd Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining), test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) Pass ================================================================= [6862769] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/uninlining} time: 01:13:33 info: http://pulpito.ceph.com/mchangir-2022-06-03_16:57:52-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6862769/ [6862770] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/uninlining} time: 01:09:00 info: http://pulpito.ceph.com/mchangir-2022-06-03_16:57:52-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6862770/ [6862772] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/uninlining} time: 01:11:15 info: http://pulpito.ceph.com/mchangir-2022-06-03_16:57:52-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6862772/ [6862773] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/uninlining} time: 01:15:11 info: http://pulpito.ceph.com/mchangir-2022-06-03_16:57:52-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6862773/