2021-01-11T07:12:58.707 INFO:root:teuthology version: 1.0.0-77817aa4 2021-01-11T07:12:58.709 INFO:teuthology.results:seed: '1859' 2021-01-11T07:29:16.880 INFO:root:teuthology version: 1.0.0-77817aa4 2021-01-11T07:29:16.914 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-01-11T08:30:18.009 INFO:teuthology.results:Tests finished! gathering results... 2021-01-11T08:30:20.309 INFO:scrape:Found 20 jobs 2021-01-11T08:30:23.061 INFO:scrape:Found 2 distinct failure reasons 2021-01-11T08:30:23.062 INFO:scrape:Failure: Test failure: test_idempotency (tasks.cephfs.test_volume_client.TestVolumeClient) 2021-01-11T08:30:23.063 INFO:scrape:3 jobs: ['5777535', '5777540', '5777530'] 2021-01-11T08:30:23.063 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs:volumes/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/volume-client}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-01-11T08:30:23.064 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'fs:volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'rhel_8}', 'tasks/volume-client}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-01-11T08:30:23.064 INFO:scrape: 2021-01-11T08:30:23.064 INFO:scrape:Failure: Test failure: test_evict_client (tasks.cephfs.test_volume_client.TestVolumeClient) 2021-01-11T08:30:23.065 INFO:scrape:1 jobs: ['5777525'] 2021-01-11T08:30:23.065 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs:volumes/{begin', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/volume-client}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-01-11T08:30:23.066 INFO:scrape: 2021-01-11T08:30:23.067 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 16 Fail ================================================================= [5777525] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} ----------------------------------------------------------------- time: 00:16:44 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777525/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777525/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8c3952da79034736872cc2ae93f23c5e Test failure: test_evict_client (tasks.cephfs.test_volume_client.TestVolumeClient) [5777530] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} ----------------------------------------------------------------- time: 00:16:24 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777530/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777530/ sentry: https://sentry.ceph.com/organizations/ceph/?query=871c5ead3b9247fabba72f306dfe894f Test failure: test_idempotency (tasks.cephfs.test_volume_client.TestVolumeClient) [5777535] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} ----------------------------------------------------------------- time: 00:24:22 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777535/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777535/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e7d39148babd42fbb497de6a8fe329f8 Test failure: test_idempotency (tasks.cephfs.test_volume_client.TestVolumeClient) [5777540] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} ----------------------------------------------------------------- time: 00:17:47 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777540/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777540/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f48f9dd43afa47c4983f0f7264f6666d Test failure: test_idempotency (tasks.cephfs.test_volume_client.TestVolumeClient) Pass ================================================================= [5777526] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:34:39 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777526/ [5777527] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:53:00 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777527/ [5777528] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:16:09 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777528/ [5777529] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:16:20 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777529/ [5777531] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:57:21 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777531/ [5777532] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:37:42 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777532/ [5777533] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:15:23 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777533/ [5777534] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:16:14 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777534/ [5777536] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:42:31 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777536/ [5777537] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:35:05 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777537/ [5777538] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:13:47 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777538/ [5777539] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:24:27 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777539/ [5777541] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:39:42 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777541/ [5777542] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:33:25 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777542/ [5777543] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:20:19 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777543/ [5777544] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:18:52 info: http://pulpito.ceph.com/pdonnell-2021-01-11_07:12:28-fs:volumes-wip-pdonnell-testing-20210111.051135-distro-basic-smithi/5777544/