2021-03-20T03:11:52.301 INFO:root:teuthology version: 1.1.0-c48eb744 2021-03-20T03:11:52.302 INFO:teuthology.results:seed: '6565' 2021-03-20T03:35:42.009 INFO:root:teuthology version: 1.1.0-c48eb744 2021-03-20T03:35:42.067 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-20T04:14:44.995 INFO:teuthology.results:Tests finished! gathering results... 2021-03-20T04:14:46.814 INFO:scrape:Found 27 jobs 2021-03-20T04:14:47.669 INFO:scrape:Found 2 distinct failure reasons 2021-03-20T04:14:47.669 INFO:scrape:Failure: Test failure: test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes), test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes) 2021-03-20T04:14:47.669 INFO:scrape:2 jobs: ['5982356', '5982355'] 2021-03-20T04:14:47.670 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs:volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/basic}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-20T04:14:47.670 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs:volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/basic}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-20T04:14:47.670 INFO:scrape: 2021-03-20T04:14:47.670 INFO:scrape:Failure: Test failure: test_subvolumegroup_pin_distributed (tasks.cephfs.test_volumes.TestSubvolumeGroups) 2021-03-20T04:14:47.670 INFO:scrape:1 jobs: ['5982357'] 2021-03-20T04:14:47.670 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs:volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/basic}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-20T04:14:47.670 INFO:scrape: 2021-03-20T04:14:47.672 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/ failed: 3 dead: 0 running: 0 waiting: 0 queued: 0 passed: 24 Fail ================================================================= [5982355] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 00:35:13 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982355/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982355/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3dffdc9487c846eababf9158be579567 Test failure: test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes), test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes) [5982356] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 00:33:22 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982356/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982356/ sentry: https://sentry.ceph.com/organizations/ceph/?query=65f6454b5b614a3da3b0f704628498a7 Test failure: test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes), test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes) [5982357] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 00:19:28 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982357/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982357/ sentry: https://sentry.ceph.com/organizations/ceph/?query=54cbbb1de4cf497a8606c2d24ac711ea Test failure: test_subvolumegroup_pin_distributed (tasks.cephfs.test_volumes.TestSubvolumeGroups) Pass ================================================================= [5982349] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:44:30 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982349/ [5982350] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:43:56 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982350/ [5982351] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:43:47 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982351/ [5982352] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:19:25 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982352/ [5982353] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:19:45 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982353/ [5982354] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:19:28 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982354/ [5982358] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:34:10 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982358/ [5982359] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:34:09 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982359/ [5982360] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:34:46 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982360/ [5982361] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:34:19 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982361/ [5982362] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:34:09 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982362/ [5982363] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:34:07 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982363/ [5982364] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:14:46 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982364/ [5982365] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:13:52 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982365/ [5982366] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:14:42 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982366/ [5982367] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:37:19 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982367/ [5982368] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:37:12 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982368/ [5982369] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:38:59 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982369/ [5982370] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:15:40 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982370/ [5982371] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:15:18 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982371/ [5982372] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:15:24 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982372/ [5982373] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:19:01 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982373/ [5982374] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:18:22 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982374/ [5982375] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:18:37 info: http://pulpito.ceph.com/pdonnell-2021-03-20_03:07:39-fs:volumes-wip-pdonnell-testing-20210319.214557-distro-basic-smithi/5982375/