2021-03-23T12:22:19.383 INFO:root:teuthology version: 1.1.0-6b3150e9 2021-03-23T12:22:19.384 INFO:teuthology.results:subset: '393/9999' 2021-03-23T12:22:19.385 INFO:teuthology.results:seed: '3216' 2021-03-23T13:00:40.517 INFO:root:teuthology version: 1.1.0-6b3150e9 2021-03-23T13:00:40.550 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-03-23T14:42:47.996 INFO:teuthology.results:Tests finished! gathering results... 2021-03-23T14:42:49.493 INFO:scrape:Found 15 jobs 2021-03-23T14:42:49.498 INFO:scrape:Found 1 distinct failure reasons 2021-03-23T14:42:49.499 INFO:scrape:Failure: "2021-03-23T13:12:54.977441+0000 mon.a (mon.0) 27 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log 2021-03-23T14:42:49.499 INFO:scrape:1 jobs: ['5990543'] 2021-03-23T14:42:49.500 INFO:scrape:suites: ['objectstore/bluestore-bitmap', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/ubuntu_latest', 'tasks/radosbench', 'thrashosds-health', 'whitelist_health}'] 2021-03-23T14:42:49.500 INFO:scrape: 2021-03-23T14:42:49.501 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 14 Fail ================================================================= [5990543] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-bitmap powercycle/default supported-all-distro/ubuntu_latest tasks/radosbench thrashosds-health whitelist_health} ----------------------------------------------------------------- time: 01:35:41 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990543/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990543/ "2021-03-23T13:12:54.977441+0000 mon.a (mon.0) 27 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log Pass ================================================================= [5990534] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-bitmap powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_kernel_untar_build thrashosds-health whitelist_health} time: 00:43:01 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990534/ [5990535] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-lz4 powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_misc thrashosds-health whitelist_health} time: 00:25:30 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990535/ [5990536] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-snappy powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_suites_ffsb thrashosds-health whitelist_health} time: 00:25:17 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990536/ [5990537] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zlib powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_suites_fsstress thrashosds-health whitelist_health} time: 00:11:19 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990537/ [5990538] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_fsx thrashosds-health whitelist_health} time: 00:16:21 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990538/ [5990539] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-hybrid powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_suites_fsync thrashosds-health whitelist_health} time: 00:14:51 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990539/ [5990540] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-low-osd-mem-target powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_suites_pjd thrashosds-health whitelist_health} time: 00:12:39 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990540/ [5990541] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-stupid powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_truncate_delay thrashosds-health whitelist_health} time: 00:09:38 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990541/ [5990542] powercycle/osd/{clusters/3osd-1per-target objectstore/filestore-xfs powercycle/default supported-all-distro/rhel_8 tasks/rados_api_tests thrashosds-health whitelist_health} time: 00:30:55 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990542/ [5990544] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-lz4 powercycle/default supported-all-distro/centos_8 tasks/readwrite thrashosds-health whitelist_health} time: 00:12:51 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990544/ [5990545] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-snappy powercycle/default supported-all-distro/rhel_8 tasks/snaps-few-objects thrashosds-health whitelist_health} time: 00:25:34 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990545/ [5990546] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zlib powercycle/default supported-all-distro/ubuntu_latest tasks/snaps-many-objects thrashosds-health whitelist_health} time: 00:26:10 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990546/ [5990547] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/centos_8 tasks/admin_socket_objecter_requests thrashosds-health whitelist_health} time: 00:14:06 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990547/ [5990548] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-hybrid powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_kernel_untar_build thrashosds-health whitelist_health} time: 00:45:47 info: http://pulpito.ceph.com/teuthology-2021-03-23_12:17:03-powercycle-pacific-distro-basic-smithi/5990548/