2022-05-24T08:57:27.671 INFO:root:teuthology version: 1.1.1.dev333+gb018fa01 2022-05-24T08:57:27.672 INFO:teuthology.results:seed: '1423' 2022-05-24T09:01:25.538 INFO:root:teuthology version: 1.1.1.dev333+gb018fa01 2022-05-24T09:01:25.545 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-05-24T10:13:31.658 INFO:teuthology.results:Tests finished! gathering results... 2022-05-24T10:13:32.059 INFO:scrape:Found 6 jobs 2022-05-24T10:13:33.125 INFO:scrape:Found 1 distinct failure reasons 2022-05-24T10:13:33.126 INFO:scrape:Crash: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining), test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2022-05-24T10:13:33.126 INFO:scrape:ceph version 17.0.0-12316-gc1c4e958 (c1c4e95880c2d6fd74820ac84829c683ace74c27) quincy (dev) 1: /lib64/libpthread.so.0(+0x12c20) [0x7fe39ea12c20] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b0) [0x7fe3a0285772] 5: /usr/lib64/ceph/libceph-common.so.2(+0x285935) [0x7fe3a0285935] 6: ceph-mds(+0x49be40) [0x55cc97a9be40] 7: (Continuation::_continue_function(int, int)+0x163) [0x55cc97aa77b3] 8: (CInode::validate_disk_state(CInode::validated_data*, MDSContext*)+0x4dc) [0x55cc97a99f7c] 9: (MDSContext::complete(int)+0x203) [0x55cc97afe1d3] 10: (Server::respond_to_request(boost::intrusive_ptr&, int)+0xf2) [0x55cc9782e892] 11: (MDCache::uninline_data_work(boost::intrusive_ptr)+0x38f) [0x55cc9791faff] 12: (MDCache::dispatch_request(boost::intrusive_ptr&)+0xc4) [0x55cc9792e084] 13: (MDSContext::complete(int)+0x203) [0x55cc97afe1d3] 14: (void finish_contexts > >(ceph::common::CephContext*, std::vector >&, int)+0x8d) [0x55cc977d2b7d] 15: (Locker::eval(CInode*, int, bool)+0x4d8) [0x55cc979ccbc8] 16: (Locker::handle_client_caps(boost::intrusive_ptr const&)+0x1c9b) [0x55cc979dc94b] 17: (Locker::dispatch(boost::intrusive_ptr const&)+0x114) [0x55cc979de744] 18: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xd54) [0x55cc977d9ca4] 19: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x7cb) [0x55cc977dc8db] 20: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55cc977dcefc] 21: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x108) [0x55cc977cbdf8] 22: (DispatchQueue::entry()+0x14fa) [0x7fe3a050fa0a] 23: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe3a05c7f61] 24: /lib64/libpthread.so.0(+0x817a) [0x7fe39ea0817a] 25: clone() 2022-05-24T10:13:33.127 INFO:scrape:4 jobs: ['6847304', '6847305', '6847301', '6847306'] 2022-05-24T10:13:33.127 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin/{0-install', 'mds', 'mon', 'osd}', 'overrides/{no_client_pidfile', 'tasks/uninlining}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-05-24T10:13:33.128 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{no_client_pidfile', 'rhel_8}', 'tasks/uninlining}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-05-24T10:13:33.128 INFO:scrape: 2022-05-24T10:13:33.129 INFO:teuthology.results:Sending results to mchangir@redhat.com: Test Run: mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 2 Fail ================================================================= [6847301] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/uninlining} ----------------------------------------------------------------- time: 00:32:37 info: http://pulpito.ceph.com/mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6847301/ log: http://qa-proxy.ceph.com/teuthology/mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6847301/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3d97f9e2e1464c4d97457ad25f166ed5 Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining), test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) [6847304] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/uninlining} ----------------------------------------------------------------- time: 00:33:15 info: http://pulpito.ceph.com/mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6847304/ log: http://qa-proxy.ceph.com/teuthology/mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6847304/ sentry: https://sentry.ceph.com/organizations/ceph/?query=75284d75466b43dea9c10e1da1f45166 Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining), test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) [6847305] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/uninlining} ----------------------------------------------------------------- time: 00:23:40 info: http://pulpito.ceph.com/mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6847305/ log: http://qa-proxy.ceph.com/teuthology/mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6847305/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f07556377ed341a4adde17bbdc66ddbf Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining), test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) [6847306] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/uninlining} ----------------------------------------------------------------- time: 00:25:08 info: http://pulpito.ceph.com/mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6847306/ log: http://qa-proxy.ceph.com/teuthology/mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6847306/ sentry: https://sentry.ceph.com/organizations/ceph/?query=55fcd25d650644cfa9520c25f35a6a91 Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining), test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) Pass ================================================================= [6847302] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/uninlining} time: 01:09:21 info: http://pulpito.ceph.com/mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6847302/ [6847303] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/uninlining} time: 01:00:02 info: http://pulpito.ceph.com/mchangir-2022-05-24_08:54:28-fs-wip-mchangir-mds-uninline-file-during-scrub-distro-default-smithi/6847303/