2024-03-08T08:09:51.362 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-03-08T08:09:51.363 INFO:teuthology.results:seed: '7204'
2024-03-08T08:10:49.009 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-03-08T08:10:49.013 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2024-03-08T08:41:51.229 INFO:teuthology.results:Tests finished! gathering results...
2024-03-08T08:41:51.610 INFO:scrape:Found 12 jobs
2024-03-08T08:41:52.601 INFO:scrape:Found 1 distinct failure reasons
2024-03-08T08:41:52.602 INFO:scrape:Failure: Test failure: test_per_client_labeled_perf_counters (tasks.cephfs.test_admin.TestLabeledPerfCounters)
2024-03-08T08:41:52.602 INFO:scrape:12 jobs: ['7585504', '7585506', '7585508', '7585503', '7585510', '7585511', '7585500', '7585505', '7585507', '7585502', '7585509', '7585501']
2024-03-08T08:41:52.602 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'osd}', 'overrides/{ignorelist_health', 'tasks/admin}']
2024-03-08T08:41:52.602 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_latest}', 'distro/{ubuntu_latest}', 'fs:functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'k-stock}', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{centos_9.stream', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'subvol_versions/create_subvol_version_v1', 'subvol_versions/create_subvol_version_v2', 'tasks/admin}']
2024-03-08T08:41:52.602 INFO:scrape:
2024-03-08T08:41:52.603 INFO:teuthology.results:Sending results to jcollin@redhat.com: Test Run: jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi
=================================================================
info:    https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/
failed:  12
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[7585500]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}
-----------------------------------------------------------------
time:   00:11:01
info:   https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585500/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585500/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2eb832a90a074cdfbc217c58c6247cc7

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585501]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}
-----------------------------------------------------------------
time:   00:11:15
info:   https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585501/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585501/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c9a0d2b12c774176a30481b7c427dbea

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585502]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}
-----------------------------------------------------------------
time:   00:20:18
info:   https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585502/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585502/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9b7e90d2598643b485ffefa26fce753c

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585503]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}
-----------------------------------------------------------------
time:   00:10:51
info:   https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585503/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585503/
sentry: https://sentry.ceph.com/organizations/ceph/?query=67eab06a2db74a23a42e2d392569a502

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585504]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}
-----------------------------------------------------------------
time:   00:18:03
info:   https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585504/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585504/
sentry: https://sentry.ceph.com/organizations/ceph/?query=edc1733d216344f5bbcae8011bde9e6f

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585505]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}
-----------------------------------------------------------------
time:   00:16:58
info:   https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585505/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585505/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1e214d22f5a74893b4db3a40690e22ce

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585506]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}
-----------------------------------------------------------------
time:   00:10:44
info:   https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585506/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cd312c27103a4b5eaf38a693cff508aa

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585507]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}
-----------------------------------------------------------------
time:   00:11:21
info:   https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585507/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585507/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8a03b79de8974df3bc5f3286f843b52d

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585508]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/admin}
-----------------------------------------------------------------
time:   00:13:43
info:   https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585508/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585508/
sentry: https://sentry.ceph.com/organizations/ceph/?query=538f41495f524635bc0f4ed6294a3727

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585509]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}
-----------------------------------------------------------------
time:   00:10:33
info:   https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585509/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585509/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ed796efec5074a48bcab63b2dc1a5fed

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585510]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}
-----------------------------------------------------------------
time:   00:12:35
info:   https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585510/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585510/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f88257a81fbd4caf8a2a4dc70d094848

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)

[7585511]  fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/admin}
-----------------------------------------------------------------
time:   00:17:10
info:   https://pulpito.ceph.com/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585511/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2024-03-08_08:09:40-fs:functional-main-distro-default-smithi/7585511/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b62e80cd3a7d4353aae51bfe3f486971

    Test failure: test_per_client_labeled_perf_counters
    (tasks.cephfs.test_admin.TestLabeledPerfCounters)