2022-08-19T12:37:52.441 INFO:root:teuthology version: 1.1.1.dev412+gb1d387f1
2022-08-19T12:37:52.443 INFO:teuthology.results:seed: '2905'
2022-08-19T12:42:05.033 INFO:root:teuthology version: 1.1.1.dev412+gb1d387f1
2022-08-19T12:42:05.038 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-08-19T13:21:08.251 INFO:teuthology.results:Tests finished! gathering results...
2022-08-19T13:21:08.876 INFO:scrape:Found 6 jobs
2022-08-19T13:21:14.918 INFO:scrape:Found 2 distinct failure reasons
2022-08-19T13:21:14.919 INFO:scrape:Failure: Test failure: test_delayed_metrics (tasks.cephfs.test_mds_metrics.TestMDSMetrics)
2022-08-19T13:21:14.920 INFO:scrape:3 jobs: ['6980529', '6980531', '6980528']
2022-08-19T13:21:14.921 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'osd}', 'overrides/{ignorelist_health', 'tasks/metrics}']
2022-08-19T13:21:14.922 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/kclient/{mount-syntax/{v1}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/metrics}']
2022-08-19T13:21:14.922 INFO:scrape:
2022-08-19T13:21:14.923 INFO:scrape:Failure: "2022-08-19T13:13:22.293581+0000 mon.a (mon.0) 2629 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log
2022-08-19T13:21:14.924 INFO:scrape:2 jobs: ['6980532', '6980533']
2022-08-19T13:21:14.924 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{ignorelist_health', 'tasks/metrics}']
2022-08-19T13:21:14.925 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'osd}', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'tasks/metrics}']
2022-08-19T13:21:14.926 INFO:scrape:
2022-08-19T13:21:14.927 INFO:teuthology.results:Sending results to jcollin@redhat.com: Test Run: jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/
failed:  5
dead:    0
running: 0
waiting: 0
queued:  0
passed:  1


Fail
=================================================================
[6980528]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics}
-----------------------------------------------------------------
time:   00:16:06
info:   http://pulpito.ceph.com/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/6980528/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/6980528/
sentry: https://sentry.ceph.com/organizations/ceph/?query=279e4b6991c34454a7425a59c0ec6641

    Test failure: test_delayed_metrics
    (tasks.cephfs.test_mds_metrics.TestMDSMetrics)

[6980529]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics}
-----------------------------------------------------------------
time:   00:15:15
info:   http://pulpito.ceph.com/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/6980529/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/6980529/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2690bd13ef97402c94494d20b462a505

    Test failure: test_delayed_metrics
    (tasks.cephfs.test_mds_metrics.TestMDSMetrics)

[6980531]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics}
-----------------------------------------------------------------
time:   00:15:27
info:   http://pulpito.ceph.com/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/6980531/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/6980531/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e4248c8156254fdc81c9392946b0fa35

    Test failure: test_delayed_metrics
    (tasks.cephfs.test_mds_metrics.TestMDSMetrics)

[6980532]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics}
-----------------------------------------------------------------
time:   00:29:20
info:   http://pulpito.ceph.com/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/6980532/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/6980532/

    "2022-08-19T13:13:22.293581+0000 mon.a (mon.0) 2629 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg inactive, 1 pg peering
    (PG_AVAILABILITY)" in cluster log

[6980533]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics}
-----------------------------------------------------------------
time:   00:29:30
info:   http://pulpito.ceph.com/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/6980533/
log:    http://qa-proxy.ceph.com/teuthology/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/6980533/

    "2022-08-19T13:11:25.442917+0000 mon.a (mon.0) 2790 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg inactive, 1 pg peering
    (PG_AVAILABILITY)" in cluster log



Pass
=================================================================
[6980530] fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/metrics}
time:   00:29:31
info:   http://pulpito.ceph.com/jcollin-2022-08-19_12:36:04-fs-wip-jcollin-perf-stats-fstop-testing_18_Aug_22-distro-default-smithi/6980530/