2021-03-19T07:13:54.004 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-19T07:13:54.005 INFO:teuthology.results:subset: '0/8' 2021-03-19T07:13:54.005 INFO:teuthology.results:seed: '7800' 2021-03-19T07:21:08.461 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-19T07:21:08.509 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-19T07:48:10.615 INFO:teuthology.results:Tests finished! gathering results... 2021-03-19T07:48:11.380 INFO:scrape:Found 14 jobs 2021-03-19T07:48:12.145 INFO:scrape:Found 2 distinct failure reasons 2021-03-19T07:48:12.146 INFO:scrape:Failure: "2021-03-19T07:37:13.167132+0000 mon.a (mon.0) 109 : cluster [ERR] Health check failed: no active mgr (MGR_DOWN)" in cluster log 2021-03-19T07:48:12.146 INFO:scrape:4 jobs: ['5980891', '5980883', '5980885', '5980884'] 2021-03-19T07:48:12.146 INFO:scrape:suites intersection: ['clusters/{fixed-2', 'mon_election/classic', 'msgr-failures/few', 'openstack}', 'rados', 'rados:basic/{ceph', 'supported-random-distro$/{ubuntu_latest}', 'tasks/libcephsqlite}'] 2021-03-19T07:48:12.146 INFO:scrape:suites union: ['clusters/{fixed-2', 'mon_election/classic', 'msgr-failures/few', 'msgr/async', 'msgr/async-v1only', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados:basic/{ceph', 'supported-random-distro$/{ubuntu_latest}', 'tasks/libcephsqlite}'] 2021-03-19T07:48:12.147 INFO:scrape: 2021-03-19T07:48:12.147 INFO:scrape:Failure: 'NoneType' object has no attribute 'get' 2021-03-19T07:48:12.147 INFO:scrape:1 jobs: ['5980886'] 2021-03-19T07:48:12.147 INFO:scrape:suites: ['clusters/{fixed-2', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados:basic/{ceph', 'supported-random-distro$/{rhel_8}', 'tasks/libcephsqlite}'] 2021-03-19T07:48:12.147 INFO:scrape: 2021-03-19T07:48:12.148 INFO:teuthology.results:Sending results to pdonnell@redhat.com: Test Run: pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/ failed: 5 dead: 0 running: 0 waiting: 0 queued: 0 passed: 9 Fail ================================================================= [5980883] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} ----------------------------------------------------------------- time: 00:11:31 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980883/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980883/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6fc58f8844f64c4080511ed6432f6df6 "2021-03-19T07:33:55.377934+0000 mon.a (mon.0) 111 : cluster [ERR] Health check failed: no active mgr (MGR_DOWN)" in cluster log [5980884] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} ----------------------------------------------------------------- time: 00:10:51 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980884/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980884/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8e8b74120b2b42b09cccedd4802147d3 "2021-03-19T07:33:59.342359+0000 mon.a (mon.0) 107 : cluster [ERR] Health check failed: no active mgr (MGR_DOWN)" in cluster log [5980885] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} ----------------------------------------------------------------- time: 00:11:31 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980885/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980885/ sentry: https://sentry.ceph.com/organizations/ceph/?query=027ac171be7b42f3980fe167c4522fbf "2021-03-19T07:33:51.339369+0000 mon.a (mon.0) 108 : cluster [ERR] Health check failed: no active mgr (MGR_DOWN)" in cluster log [5980886] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/libcephsqlite} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980886/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980886/ sentry: https://sentry.ceph.com/organizations/ceph/?query=58dbe3c1717f49629f966706200365c2 'NoneType' object has no attribute 'get' [5980891] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} ----------------------------------------------------------------- time: 00:10:54 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980891/ log: http://qa-proxy.ceph.com/teuthology/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980891/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8b1bbd9e908e4c72a80d4f664f1ea170 "2021-03-19T07:37:13.167132+0000 mon.a (mon.0) 109 : cluster [ERR] Health check failed: no active mgr (MGR_DOWN)" in cluster log Pass ================================================================= [5980881] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:18:16 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980881/ [5980882] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:18:43 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980882/ [5980887] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:16:51 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980887/ [5980888] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:18:19 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980888/ [5980889] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:18:04 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980889/ [5980890] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/libcephsqlite} time: 00:19:27 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980890/ [5980892] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:16:50 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980892/ [5980893] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:12:59 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980893/ [5980894] rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/libcephsqlite} time: 00:17:45 info: http://pulpito.ceph.com/pdonnell-2021-03-19_07:13:39-rados:basic-wip-pdonnell-testing-20210319.041258-distro-basic-smithi/5980894/