2021-09-22T12:17:36.375 INFO:root:teuthology version: 1.1.0-d3a488f5 2021-09-22T12:17:36.376 INFO:teuthology.results:subset: '4480/9999' 2021-09-22T12:17:36.376 INFO:teuthology.results:seed: '3094' 2021-09-22T12:26:37.276 INFO:root:teuthology version: 1.1.0-d3a488f5 2021-09-22T12:26:37.287 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-09-22T13:49:42.443 INFO:teuthology.results:Tests finished! gathering results... 2021-09-22T13:49:43.321 INFO:scrape:Found 15 jobs 2021-09-22T13:49:43.322 INFO:scrape:Found 1 distinct failure reasons 2021-09-22T13:49:43.323 INFO:scrape:Failure: "2021-09-22T12:44:03.521972+0000 mon.a (mon.0) 23 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log 2021-09-22T13:49:43.323 INFO:scrape:1 jobs: ['6402586'] 2021-09-22T13:49:43.324 INFO:scrape:suites: ['objectstore/bluestore-comp-lz4', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/ubuntu_latest', 'tasks/snaps-many-objects', 'thrashosds-health', 'whitelist_health}'] 2021-09-22T13:49:43.324 INFO:scrape: 2021-09-22T13:49:43.324 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 14 Fail ================================================================= [6402586] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-lz4 powercycle/default supported-all-distro/ubuntu_latest tasks/snaps-many-objects thrashosds-health whitelist_health} ----------------------------------------------------------------- time: 00:29:41 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402586/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402586/ "2021-09-22T12:44:03.521972+0000 mon.a (mon.0) 23 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log Pass ================================================================= [6402573] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-low-osd-mem-target powercycle/default supported-all-distro/rhel_8 tasks/admin_socket_objecter_requests thrashosds-health whitelist_health} time: 00:18:46 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402573/ [6402574] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-stupid powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_kernel_untar_build thrashosds-health whitelist_health} time: 00:49:00 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402574/ [6402575] powercycle/osd/{clusters/3osd-1per-target objectstore/filestore-xfs powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_misc thrashosds-health whitelist_health} time: 00:21:13 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402575/ [6402576] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-bitmap powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_suites_ffsb thrashosds-health whitelist_health} time: 00:30:40 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402576/ [6402577] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-lz4 powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_suites_fsstress thrashosds-health whitelist_health} time: 00:14:02 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402577/ [6402578] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-snappy powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_fsx thrashosds-health whitelist_health} time: 00:17:29 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402578/ [6402579] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zlib powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_suites_fsync thrashosds-health whitelist_health} time: 00:17:55 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402579/ [6402580] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_suites_pjd thrashosds-health whitelist_health} time: 00:12:40 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402580/ [6402581] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-hybrid powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_truncate_delay thrashosds-health whitelist_health} time: 00:10:48 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402581/ [6402582] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-low-osd-mem-target powercycle/default supported-all-distro/rhel_8 tasks/rados_api_tests thrashosds-health whitelist_health} time: 00:32:41 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402582/ [6402583] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-stupid powercycle/default supported-all-distro/ubuntu_latest tasks/radosbench thrashosds-health whitelist_health} time: 01:15:24 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402583/ [6402584] powercycle/osd/{clusters/3osd-1per-target objectstore/filestore-xfs powercycle/default supported-all-distro/centos_8 tasks/readwrite thrashosds-health whitelist_health} time: 00:14:51 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402584/ [6402585] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-bitmap powercycle/default supported-all-distro/rhel_8 tasks/snaps-few-objects thrashosds-health whitelist_health} time: 00:31:48 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402585/ [6402587] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-snappy powercycle/default supported-all-distro/centos_8 tasks/admin_socket_objecter_requests thrashosds-health whitelist_health} time: 00:18:40 info: http://pulpito.ceph.com/teuthology-2021-09-22_12:17:02-powercycle-pacific-distro-basic-smithi/6402587/