2021-03-19T05:27:55.427 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-19T05:27:55.428 INFO:teuthology.results:subset: '3/4' 2021-03-19T05:27:55.428 INFO:teuthology.results:seed: '7113' 2021-03-19T06:00:22.599 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-19T06:00:22.632 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-19T06:49:26.398 INFO:teuthology.results:Tests finished! gathering results... 2021-03-19T06:49:28.867 INFO:scrape:Found 30 jobs 2021-03-19T06:49:30.939 INFO:scrape:Found 3 distinct failure reasons 2021-03-19T06:49:30.939 INFO:scrape:Failure: Test failure: test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes), test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes) 2021-03-19T06:49:30.939 INFO:scrape:2 jobs: ['5980849', '5980851'] 2021-03-19T06:49:30.940 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/basic}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-19T06:49:30.940 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/basic}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-19T06:49:30.940 INFO:scrape: 2021-03-19T06:49:30.940 INFO:scrape:Failure: Command failed on smithi205 with status 123: "find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" 2021-03-19T06:49:30.940 INFO:scrape:1 jobs: ['5980868'] 2021-03-19T06:49:30.940 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/clone}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-19T06:49:30.940 INFO:scrape: 2021-03-19T06:49:30.940 INFO:scrape:Failure: Test failure: test_subvolumegroup_pin_distributed (tasks.cephfs.test_volumes.TestSubvolumeGroups) 2021-03-19T06:49:30.941 INFO:scrape:1 jobs: ['5980850'] 2021-03-19T06:49:30.941 INFO:scrape:suites: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'fs/volumes/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'no_client_pidfile', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{frag_enable', 'tasks/volumes/{overrides', 'test/basic}}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-03-19T06:49:30.941 INFO:scrape: 2021-03-19T06:49:30.943 INFO:teuthology.results:Sending results to khiremat@redhat.com: Test Run: khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 26 Fail ================================================================= [5980849] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 00:40:13 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980849/ log: http://qa-proxy.ceph.com/teuthology/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980849/ sentry: https://sentry.ceph.com/organizations/ceph/?query=09f1bc5dadca4c6c888416d3fe8113be Test failure: test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes), test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes) [5980850] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 00:21:53 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980850/ log: http://qa-proxy.ceph.com/teuthology/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980850/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c1ba9d17f68249b5a78c2ba08c034bdc Test failure: test_subvolumegroup_pin_distributed (tasks.cephfs.test_volumes.TestSubvolumeGroups) [5980851] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} ----------------------------------------------------------------- time: 00:39:30 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980851/ log: http://qa-proxy.ceph.com/teuthology/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980851/ sentry: https://sentry.ceph.com/organizations/ceph/?query=afc34a78fa3b40af939a9b16a3d96c73 Test failure: test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes), test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes) [5980868] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} ----------------------------------------------------------------- time: 00:42:30 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980868/ log: http://qa-proxy.ceph.com/teuthology/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980868/ Command failed on smithi205 with status 123: "find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" Pass ================================================================= [5980852] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:35:38 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980852/ [5980853] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:34:57 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980853/ [5980854] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:34:54 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980854/ [5980855] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:13:27 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980855/ [5980856] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:13:09 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980856/ [5980857] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:13:13 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980857/ [5980858] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs} time: 00:12:00 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980858/ [5980859] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs} time: 00:11:46 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980859/ [5980860] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs} time: 00:12:34 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980860/ [5980861] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:18:29 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980861/ [5980862] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:18:54 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980862/ [5980863] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:18:39 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980863/ [5980864] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:37:09 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980864/ [5980865] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:36:56 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980865/ [5980866] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/basic}} time: 00:36:56 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980866/ [5980867] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:44:15 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980867/ [5980869] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/clone}} time: 00:42:30 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980869/ [5980870] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python} time: 00:08:35 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980870/ [5980871] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python} time: 00:08:10 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980871/ [5980872] fs/libcephfs/{begin clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/{centos_8} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/libcephfs_python} time: 00:08:06 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980872/ [5980873] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:15:42 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980873/ [5980874] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:15:23 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980874/ [5980875] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/misc}} time: 00:15:10 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980875/ [5980876] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:16:38 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980876/ [5980877] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:16:55 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980877/ [5980878] fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volumes/{overrides test/snapshot}} time: 00:16:18 info: http://pulpito.ceph.com/khiremat-2021-03-19_05:25:43-fs-wip-khiremat-40214-retain-suid-subvol-clone-1-distro-basic-smithi/5980878/