2022-12-03T15:56:56.689 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf
2022-12-03T15:56:56.690 INFO:teuthology.results:seed: '8350'
2022-12-03T16:33:45.294 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf
2022-12-03T16:33:45.328 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-12-03T17:40:50.246 INFO:teuthology.results:Tests finished! gathering results...
2022-12-03T17:40:50.693 INFO:scrape:Found 6 jobs
2022-12-03T17:40:53.565 INFO:scrape:Found 1 distinct failure reasons
2022-12-03T17:40:53.566 INFO:scrape:Failure: "1670086967.2502298 mon.a (mon.0) 495 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log
2022-12-03T17:40:53.567 INFO:scrape:6 jobs: ['7102024', '7102023', '7102021', '7102022', '7102020', '7102025']
2022-12-03T17:40:53.568 INFO:scrape:suites intersection: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'no_client_pidfile}', 'osd}', 'overrides/{ignorelist_health', 'tasks/snap-schedule}']
2022-12-03T17:40:53.568 INFO:scrape:suites union: ['1-ceph', '2-logrotate}', 'clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8}', 'distro/{rhel_8}', 'distro/{ubuntu_latest}', 'fs/functional/{begin/{0-install', 'ignorelist_wrongly_marked_down', 'mds', 'mon', 'mount', 'mount/fuse', 'mount/kclient/{mount-syntax/{v1}', 'mount/kclient/{mount-syntax/{v2}', 'ms-die-on-skipped}}', 'no_client_pidfile}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/k-testing', 'overrides/{ignorelist_health', 'rhel_8}', 'tasks/snap-schedule}']
2022-12-03T17:40:53.569 INFO:scrape:
2022-12-03T17:40:53.570 INFO:teuthology.results:Sending results to mchangir@redhat.com: Test Run: mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/
failed:  6
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[7102020]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   00:49:01
info:   http://pulpito.ceph.com/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/7102020/
log:    http://qa-proxy.ceph.com/teuthology/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/7102020/

    "1670084165.61931 mon.a (mon.0) 440 : cluster [WRN] Health check failed:
    Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log

[7102021]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   00:56:35
info:   http://pulpito.ceph.com/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/7102021/
log:    http://qa-proxy.ceph.com/teuthology/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/7102021/

    "1670087317.7051497 mon.a (mon.0) 560 : cluster [WRN] Health check failed:
    Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log

[7102022]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   00:48:49
info:   http://pulpito.ceph.com/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/7102022/
log:    http://qa-proxy.ceph.com/teuthology/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/7102022/

    "1670086063.8862414 mon.a (mon.0) 515 : cluster [WRN] Health check failed:
    Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log

[7102023]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   00:46:37
info:   http://pulpito.ceph.com/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/7102023/
log:    http://qa-proxy.ceph.com/teuthology/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/7102023/

    "1670087054.8033743 mon.a (mon.0) 551 : cluster [WRN] Health check failed:
    Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log

[7102024]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   00:56:42
info:   http://pulpito.ceph.com/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/7102024/
log:    http://qa-proxy.ceph.com/teuthology/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/7102024/

    "1670086967.2502298 mon.a (mon.0) 495 : cluster [WRN] Health check failed:
    Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log

[7102025]  fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}
-----------------------------------------------------------------
time:   00:49:06
info:   http://pulpito.ceph.com/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/7102025/
log:    http://qa-proxy.ceph.com/teuthology/mchangir-2022-12-03_15:49:29-fs-wip-mchangir-qa-add-mgr-thrasher-distro-default-smithi/7102025/

    "1670087119.3658278 mon.a (mon.0) 530 : cluster [WRN] Health check failed:
    Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log