2022-06-09T06:59:08.462 INFO:root:teuthology version: 1.1.1.dev370+g2290146e
2022-06-09T06:59:08.464 INFO:teuthology.results:seed: '2590'
2022-06-09T06:59:16.251 INFO:root:teuthology version: 1.1.1.dev370+g2290146e
2022-06-09T06:59:16.252 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-06-09T08:01:20.157 INFO:teuthology.results:Tests finished! gathering results...
2022-06-09T08:01:20.656 INFO:scrape:Found 12 jobs
2022-06-09T08:01:22.583 INFO:scrape:Found 3 distinct failure reasons
2022-06-09T08:01:22.584 INFO:scrape:Failure: "2022-06-09T07:29:11.896270+0000 mon.a (mon.0) 3218 : cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in cluster log
2022-06-09T08:01:22.584 INFO:scrape:1 jobs: ['6869694']
2022-06-09T08:01:22.585 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs:volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/volumes/{overrides', 'test/snapshot}}']
2022-06-09T08:01:22.585 INFO:scrape:
2022-06-09T08:01:22.586 INFO:scrape:Failure: Test failure: test_subvolumegroup_pin_distributed (tasks.cephfs.test_volumes.TestSubvolumeGroups)
2022-06-09T08:01:22.586 INFO:scrape:1 jobs: ['6869699']
2022-06-09T08:01:22.587 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'fs:volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/volumes/{overrides', 'test/basic}}']
2022-06-09T08:01:22.587 INFO:scrape:
2022-06-09T08:01:22.588 INFO:scrape:Failure: Test failure: test_subvolume_rm_force (tasks.cephfs.test_volumes.TestSubvolumes)
2022-06-09T08:01:22.588 INFO:scrape:1 jobs: ['6869691']
2022-06-09T08:01:22.589 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs:volumes/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/volumes/{overrides', 'test/basic}}']
2022-06-09T08:01:22.589 INFO:scrape:
2022-06-09T08:01:22.590 INFO:teuthology.results:Sending results to ridave@redhat.com,rishabddave@gmail.com: Test Run: rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/
failed:  3
dead:    0
running: 0
waiting: 0
queued:  0
passed:  9


Fail
=================================================================
[6869691]  fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}
-----------------------------------------------------------------
time:   00:53:10
info:   http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869691/
log:    http://qa-proxy.ceph.com/teuthology/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869691/
sentry: https://sentry.ceph.com/organizations/ceph/?query=51d2627f8b204d38868c949c2180bb95

    Test failure: test_subvolume_rm_force
    (tasks.cephfs.test_volumes.TestSubvolumes)

[6869694]  fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}
-----------------------------------------------------------------
time:   00:28:02
info:   http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869694/
log:    http://qa-proxy.ceph.com/teuthology/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869694/

    "2022-06-09T07:29:11.896270+0000 mon.a (mon.0) 3218 : cluster [WRN] Health
    check failed: Degraded data redundancy: 1 pg degraded (PG_DEGRADED)" in
    cluster log

[6869699]  fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}
-----------------------------------------------------------------
time:   00:25:31
info:   http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869699/
log:    http://qa-proxy.ceph.com/teuthology/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869699/
sentry: https://sentry.ceph.com/organizations/ceph/?query=81b027aff3e14016ab44a8f0171bb440

    Test failure: test_subvolumegroup_pin_distributed
    (tasks.cephfs.test_volumes.TestSubvolumeGroups)



Pass
=================================================================
[6869692] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}}
time:   00:42:42
info:   http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869692/

[6869693] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}
time:   00:18:35
info:   http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869693/

[6869695] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}
time:   00:56:34
info:   http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869695/

[6869696] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}}
time:   00:50:20
info:   http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869696/

[6869697] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}
time:   00:13:44
info:   http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869697/

[6869698] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}
time:   00:26:12
info:   http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869698/

[6869700] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}}
time:   00:43:20
info:   http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869700/

[6869701] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}
time:   00:15:14
info:   http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869701/

[6869702] fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}
time:   00:31:50
info:   http://pulpito.ceph.com/rishabh-2022-06-09_06:59:00-fs:volumes-main-distro-default-smithi/6869702/