2023-11-29T09:42:44.958 INFO:root:teuthology version: 0.0.1.dev212+g735a43c
2023-11-29T09:42:44.958 INFO:teuthology.results:seed: '3233'
2023-11-29T10:11:43.115 INFO:root:teuthology version: 0.0.1.dev212+g735a43c
2023-11-29T10:11:43.118 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-11-29T12:19:52.042 INFO:teuthology.results:Tests finished! gathering results...
2023-11-29T12:19:52.279 INFO:scrape:Found 6 jobs
2023-11-29T12:19:57.003 INFO:scrape:Found 4 distinct failure reasons
2023-11-29T12:19:57.003 INFO:scrape:Failure: Test failure: test_snap_schedule_subvol_and_group_arguments_01 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments)
2023-11-29T12:19:57.003 INFO:scrape:3 jobs: ['7470596', '7470597', '7470595']
2023-11-29T12:19:57.004 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{ignorelist_health', 'tasks/snap-schedule}']
2023-11-29T12:19:57.004 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_latest}', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/snap-schedule}']
2023-11-29T12:19:57.004 INFO:scrape:
2023-11-29T12:19:57.004 INFO:scrape:Failure: Test failure: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules)
2023-11-29T12:19:57.004 INFO:scrape:1 jobs: ['7470600']
2023-11-29T12:19:57.004 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/snap-schedule}']
2023-11-29T12:19:57.004 INFO:scrape:
2023-11-29T12:19:57.004 INFO:scrape:Failure: Test failure: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir), test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir)
2023-11-29T12:19:57.005 INFO:scrape:1 jobs: ['7470598']
2023-11-29T12:19:57.005 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount/fuse', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/snap-schedule}']
2023-11-29T12:19:57.005 INFO:scrape:
2023-11-29T12:19:57.005 INFO:scrape:Failure: Test failure: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules), test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules)
2023-11-29T12:19:57.005 INFO:scrape:1 jobs: ['7470599']
2023-11-29T12:19:57.005 INFO:scrape:suites: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/snap-schedule}']
2023-11-29T12:19:57.005 INFO:scrape:
2023-11-29T12:19:57.006 INFO:teuthology.results:Sending results to mchangir@redhat.com: Test Run: mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi
=================================================================
info:    https://pulpito.ceph.com/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/
failed:  6
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[7470595]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   01:54:37
info:   https://pulpito.ceph.com/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470595/
log:    http://qa-proxy.ceph.com/teuthology/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470595/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fb1165e6b4484ba0aaac467a2b9ab586

    Test failure: test_snap_schedule_subvol_and_group_arguments_01
    (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments)

[7470596]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   01:59:01
info:   https://pulpito.ceph.com/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470596/
log:    http://qa-proxy.ceph.com/teuthology/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470596/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0da862f857e8463a9badfc4b76782a2e

    Test failure: test_snap_schedule_subvol_and_group_arguments_01
    (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments)

[7470597]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   01:58:41
info:   https://pulpito.ceph.com/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470597/
log:    http://qa-proxy.ceph.com/teuthology/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470597/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f208f34b16e241a4a8619a67af1598cb

    Test failure: test_snap_schedule_subvol_and_group_arguments_01
    (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments)

[7470598]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   01:51:45
info:   https://pulpito.ceph.com/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470598/
log:    http://qa-proxy.ceph.com/teuthology/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470598/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cb8547f009dd48df890ad4541d4cf468

    Test failure: test_snap_dir_name
    (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir),
    test_snap_dir_name
    (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir)

[7470599]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   01:45:01
info:   https://pulpito.ceph.com/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470599/
log:    http://qa-proxy.ceph.com/teuthology/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470599/
sentry: https://sentry.ceph.com/organizations/ceph/?query=82f779a2c86348998977a6130186e794

    Test failure: test_snap_schedule_with_mgr_restart
    (tasks.cephfs.test_snap_schedules.TestSnapSchedules),
    test_snap_schedule_with_mgr_restart
    (tasks.cephfs.test_snap_schedules.TestSnapSchedules)

[7470600]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   01:50:53
info:   https://pulpito.ceph.com/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470600/
log:    http://qa-proxy.ceph.com/teuthology/mchangir-2023-11-29_09:41:59-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7470600/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cce233ebd7e14336907545a0a136e46d

    Test failure: test_snap_schedule_with_mgr_restart
    (tasks.cephfs.test_snap_schedules.TestSnapSchedules)