2020-11-03T02:27:01.076 INFO:root:teuthology version: 1.0.0-41b569f 2020-11-03T02:27:01.077 INFO:teuthology.results:seed: '1949' 2020-11-03T10:17:07.503 INFO:root:teuthology version: 1.0.0-41b569f 2020-11-03T10:17:08.794 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-11-03T10:59:09.540 INFO:teuthology.results:Tests finished! gathering results... 2020-11-03T10:59:12.148 INFO:scrape:Found 20 jobs 2020-11-03T10:59:20.541 INFO:scrape:Found 4 distinct failure reasons 2020-11-03T10:59:20.543 INFO:scrape:Failure: Test failure: test_subvolume_snapshot_clone_pool_layout (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2020-11-03T10:59:20.543 INFO:scrape:2 jobs: ['5585374', '5585379'] 2020-11-03T10:59:20.544 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs:volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/clone}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-03T10:59:20.545 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'fs:volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/clone}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-03T10:59:20.546 INFO:scrape: 2020-11-03T10:59:20.546 INFO:scrape:Failure: Test failure: test_subvolumegroup_pin_distributed (tasks.cephfs.test_volumes.TestSubvolumeGroups) 2020-11-03T10:59:20.547 INFO:scrape:1 jobs: ['5585378'] 2020-11-03T10:59:20.547 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs:volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/basic}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-03T10:59:20.548 INFO:scrape: 2020-11-03T10:59:20.549 INFO:scrape:Failure: Test failure: test_subvolume_snapshot_clone_cancel_in_progress (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2020-11-03T10:59:20.549 INFO:scrape:1 jobs: ['5585389'] 2020-11-03T10:59:20.551 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs:volumes/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{frag_enable', 'rhel_8}', 'tasks/volumes/{overrides', 'test/clone}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-03T10:59:20.552 INFO:scrape: 2020-11-03T10:59:20.553 INFO:scrape:Failure: Test failure: test_subvolume_resize_fail_quota_lt_used_size_no_shrink (tasks.cephfs.test_volumes.TestSubvolumes) 2020-11-03T10:59:20.553 INFO:scrape:2 jobs: ['5585373', '5585383'] 2020-11-03T10:59:20.554 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs:volumes/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/basic}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-03T10:59:20.555 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs:volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/ubuntu_latest', 'overrides/{frag_enable', 'rhel_8}', 'tasks/volumes/{overrides', 'test/basic}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2020-11-03T10:59:20.555 INFO:scrape: 2020-11-03T10:59:20.557 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/ failed: 6 dead: 0 running: 0 waiting: 0 queued: 0 passed: 14 Fail ================================================================= [5585373] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 00:32:50 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585373/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585373/ sentry: https://sentry.ceph.com/ceph/sepia/?query=5d27bb5f4bde46e6b5aae1f5c1983a85 Test failure: test_subvolume_resize_fail_quota_lt_used_size_no_shrink (tasks.cephfs.test_volumes.TestSubvolumes) [5585374] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} ----------------------------------------------------------------- time: 00:37:32 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585374/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585374/ sentry: https://sentry.ceph.com/ceph/sepia/?query=1b46514565b14e0ebd5a2021781393f3 Test failure: test_subvolume_snapshot_clone_pool_layout (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) [5585378] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 00:20:55 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585378/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585378/ sentry: https://sentry.ceph.com/ceph/sepia/?query=e195d20913d04cafa1b256675bed41f4 Test failure: test_subvolumegroup_pin_distributed (tasks.cephfs.test_volumes.TestSubvolumeGroups) [5585379] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} ----------------------------------------------------------------- time: 00:30:31 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585379/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585379/ sentry: https://sentry.ceph.com/ceph/sepia/?query=e9f30c0a71374b13ad4dcdc36990142f Test failure: test_subvolume_snapshot_clone_pool_layout (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) [5585383] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 00:32:58 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585383/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585383/ sentry: https://sentry.ceph.com/ceph/sepia/?query=0de6f38d74aa44d08d270dada706a488 Test failure: test_subvolume_resize_fail_quota_lt_used_size_no_shrink (tasks.cephfs.test_volumes.TestSubvolumes) [5585389] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} ----------------------------------------------------------------- time: 00:25:06 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585389/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585389/ sentry: https://sentry.ceph.com/ceph/sepia/?query=ba0d604ebb834a33b8a67d93d57d927b Test failure: test_subvolume_snapshot_clone_cancel_in_progress (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) Pass ================================================================= [5585372] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} time: 00:23:08 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585372/ [5585375] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:14:06 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585375/ [5585376] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:17:36 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585376/ [5585377] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} time: 00:20:31 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585377/ [5585380] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:11:44 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585380/ [5585381] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:17:19 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585381/ [5585382] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} time: 00:24:30 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585382/ [5585384] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:31:18 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585384/ [5585385] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:16:16 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585385/ [5585386] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:19:17 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585386/ [5585387] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} time: 00:21:41 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585387/ [5585388] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:33:46 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585388/ [5585390] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:15:07 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585390/ [5585391] fs:volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:16:39 info: http://pulpito.ceph.com/pdonnell-2020-11-03_02:22:55-fs:volumes-wip-pdonnell-testing-20201102.231317-distro-basic-smithi/5585391/