2024-03-08T09:20:24.951 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-03-08T09:20:24.952 INFO:teuthology.results:seed: '9894'
2024-03-08T09:21:13.008 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-03-08T09:21:13.008 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2024-03-08T09:48:14.950 INFO:teuthology.results:Tests finished! gathering results...
2024-03-08T09:48:15.355 INFO:scrape:Found 12 jobs
2024-03-08T09:48:16.644 INFO:scrape:Found 1 distinct failure reasons
2024-03-08T09:48:16.644 INFO:scrape:Failure: Test failure: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters)
2024-03-08T09:48:16.644 INFO:scrape:12 jobs: ['7585518', '7585517', '7585523', '7585514', '7585525', '7585515', '7585519', '7585521', '7585516', '7585524', '7585520', '7585522']
2024-03-08T09:48:16.644 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'osd}', 'overrides/{ignorelist_health', 'tasks/admin}']
2024-03-08T09:48:16.644 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_latest}', 'distro/{ubuntu_latest}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'k-stock}', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{centos_9.stream', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'subvol_versions/create_subvol_version_v1', 'subvol_versions/create_subvol_version_v2', 'tasks/admin}']
2024-03-08T09:48:16.645 INFO:scrape:
2024-03-08T09:48:16.645 INFO:teuthology.results:Sending results to jcollin@redhat.com: Test Run: jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi
=================================================================
info:    https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/
failed:  12
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[7585514]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}
-----------------------------------------------------------------
time:   00:11:38
info:   https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585514/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585514/
sentry: https://sentry.ceph.com/organizations/ceph/?query=01e6ebed9f2445e49eae92d18b0e2d48

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585515]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}
-----------------------------------------------------------------
time:   00:10:12
info:   https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585515/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585515/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a6a15f9aa6ff40c3bb2f68a881278535

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585516]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}
-----------------------------------------------------------------
time:   00:14:12
info:   https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585516/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585516/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bfe4c63c04e54180885d22ab24f91a77

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585517]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}
-----------------------------------------------------------------
time:   00:10:12
info:   https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585517/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585517/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fbcd204ce28644b8af7dc2643952eb0b

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585518]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}
-----------------------------------------------------------------
time:   00:14:27
info:   https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585518/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585518/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b3aadd9174a443229504afa2dcca971d

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585519]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}
-----------------------------------------------------------------
time:   00:14:50
info:   https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585519/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585519/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4bc5df2e8d5e4559a67b5e7886fd7250

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585520]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}
-----------------------------------------------------------------
time:   00:09:42
info:   https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585520/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585520/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e0f265f442fa4fe6b1909f63a112e3d1

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585521]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}
-----------------------------------------------------------------
time:   00:14:03
info:   https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585521/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585521/
sentry: https://sentry.ceph.com/organizations/ceph/?query=28e4ebd675b544cba25c8d9f1195f470

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585522]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}
-----------------------------------------------------------------
time:   00:12:37
info:   https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585522/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585522/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ec28ae544e714c3b810419e171c21302

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585523]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}
-----------------------------------------------------------------
time:   00:12:07
info:   https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585523/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585523/
sentry: https://sentry.ceph.com/organizations/ceph/?query=90283c82316746b4b456ea144287128d

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585524]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}
-----------------------------------------------------------------
time:   00:14:04
info:   https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585524/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585524/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cf2dfc102db848548f4e81ce25cdecd0

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585525]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}
-----------------------------------------------------------------
time:   00:13:25
info:   https://pulpito.ceph.com/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585525/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_09:20:04-fs:functional-main-distro-default-smithi/7585525/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a5432d944d0b4900ae0664fa24097909

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)