2021-10-23T02:59:25.936 INFO:root:teuthology version: 1.1.0-c56135d1 2021-10-23T02:59:25.937 INFO:teuthology.results:subset: '3/4' 2021-10-23T02:59:25.938 INFO:teuthology.results:seed: '9370' 2021-10-23T02:59:27.460 INFO:root:teuthology version: 1.1.0-c56135d1 2021-10-23T02:59:27.462 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-10-23T03:46:30.067 INFO:teuthology.results:Tests finished! gathering results... 2021-10-23T03:46:30.158 INFO:scrape:Found 2 jobs 2021-10-23T03:46:30.693 INFO:scrape:Found 1 distinct failure reasons 2021-10-23T03:46:30.694 INFO:scrape:Failure: "2021-10-23T03:18:43.127023+0000 mon.a (mon.0) 323 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log 2021-10-23T03:46:30.695 INFO:scrape:2 jobs: ['6458021', '6458022'] 2021-10-23T03:46:30.695 INFO:scrape:suites intersection: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'fs/functional/{begin', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'osd}', 'overrides/{no_client_pidfile', 'tasks/snap-schedule}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-10-23T03:46:30.696 INFO:scrape:suites union: ['clusters/1a3s-mds-4c-client', 'conf/{client', 'distro/{centos_8.stream}', 'distro/{rhel_8}', 'fs/functional/{begin', 'k-testing}', 'mds', 'mon', 'mount/kclient/{mount', 'ms-die-on-skipped}}', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-ec-root', 'osd}', 'overrides/{distro/stock/{k-stock', 'overrides/{distro/testing/{flavor/centos_latest', 'overrides/{no_client_pidfile', 'rhel_8}', 'tasks/snap-schedule}', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-10-23T03:46:30.696 INFO:scrape: 2021-10-23T03:46:30.697 INFO:teuthology.results:Sending results to mchangir@redhat.com: Test Run: mchangir-2021-10-23_02:59:20-fs-wip-mchangir-mgr-snap_schedule-fix-db-connection-concurrent-usage-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/mchangir-2021-10-23_02:59:20-fs-wip-mchangir-mgr-snap_schedule-fix-db-connection-concurrent-usage-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/mchangir-2021-10-23_02:59:20-fs-wip-mchangir-mgr-snap_schedule-fix-db-connection-concurrent-usage-distro-basic-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 0 Fail ================================================================= [6458021] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:32:15 info: http://pulpito.ceph.com/mchangir-2021-10-23_02:59:20-fs-wip-mchangir-mgr-snap_schedule-fix-db-connection-concurrent-usage-distro-basic-smithi/6458021/ log: http://qa-proxy.ceph.com/teuthology/mchangir-2021-10-23_02:59:20-fs-wip-mchangir-mgr-snap_schedule-fix-db-connection-concurrent-usage-distro-basic-smithi/6458021/ "2021-10-23T03:18:43.127023+0000 mon.a (mon.0) 323 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log [6458022] fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8.stream} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/snap-schedule} ----------------------------------------------------------------- time: 00:34:12 info: http://pulpito.ceph.com/mchangir-2021-10-23_02:59:20-fs-wip-mchangir-mgr-snap_schedule-fix-db-connection-concurrent-usage-distro-basic-smithi/6458022/ log: http://qa-proxy.ceph.com/teuthology/mchangir-2021-10-23_02:59:20-fs-wip-mchangir-mgr-snap_schedule-fix-db-connection-concurrent-usage-distro-basic-smithi/6458022/ "2021-10-23T03:44:26.735384+0000 mon.a (mon.0) 1404 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log