2022-08-11T01:51:02.639 INFO:root:teuthology version: 1.1.1.dev409+g9e7483cc 2022-08-11T01:51:02.641 INFO:teuthology.results:subset: '9744/9999' 2022-08-11T01:51:02.641 INFO:teuthology.results:seed: '2115' 2022-08-11T15:32:58.463 INFO:root:teuthology version: 1.1.1.dev409+g9e7483cc 2022-08-11T15:32:58.490 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-08-11T16:50:04.740 INFO:teuthology.results:Tests finished! gathering results... 2022-08-11T16:50:05.968 INFO:scrape:Found 15 jobs 2022-08-11T16:50:06.678 INFO:scrape:Found 1 distinct failure reasons 2022-08-11T16:50:06.680 INFO:scrape:Failure: "2022-08-11T15:29:45.698769+0000 mon.a (mon.0) 23 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log 2022-08-11T16:50:06.680 INFO:scrape:1 jobs: ['6965685'] 2022-08-11T16:50:06.681 INFO:scrape:suites: ['objectstore/bluestore-low-osd-mem-target', 'powercycle/default', 'powercycle/osd/{clusters/3osd-1per-target', 'supported-all-distro/rhel_8', 'tasks/cfuse_workunit_suites_fsync', 'thrashosds-health', 'whitelist_health}'] 2022-08-11T16:50:06.682 INFO:scrape: 2022-08-11T16:50:06.683 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 14 Fail ================================================================= [6965685] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-low-osd-mem-target powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_suites_fsync thrashosds-health whitelist_health} ----------------------------------------------------------------- time: 00:15:37 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965685/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965685/ "2022-08-11T15:29:45.698769+0000 mon.a (mon.0) 23 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log Pass ================================================================= [6965679] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-bitmap powercycle/default supported-all-distro/rhel_8 tasks/admin_socket_objecter_requests thrashosds-health whitelist_health} time: 00:18:10 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965679/ [6965680] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-lz4 powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_kernel_untar_build thrashosds-health whitelist_health} time: 00:47:22 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965680/ [6965681] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-snappy powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_misc thrashosds-health whitelist_health} time: 00:25:28 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965681/ [6965682] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zlib powercycle/default supported-all-distro/rhel_8 tasks/cfuse_workunit_suites_ffsb thrashosds-health whitelist_health} time: 00:27:41 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965682/ [6965683] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_suites_fsstress thrashosds-health whitelist_health} time: 00:15:10 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965683/ [6965684] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-hybrid powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_fsx thrashosds-health whitelist_health} time: 00:19:50 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965684/ [6965686] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-stupid powercycle/default supported-all-distro/ubuntu_latest tasks/cfuse_workunit_suites_pjd thrashosds-health whitelist_health} time: 00:16:18 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965686/ [6965687] powercycle/osd/{clusters/3osd-1per-target objectstore/filestore-xfs powercycle/default supported-all-distro/centos_8 tasks/cfuse_workunit_suites_truncate_delay thrashosds-health whitelist_health} time: 00:14:29 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965687/ [6965688] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-bitmap powercycle/default supported-all-distro/rhel_8 tasks/rados_api_tests thrashosds-health whitelist_health} time: 00:34:34 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965688/ [6965689] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-lz4 powercycle/default supported-all-distro/ubuntu_latest tasks/radosbench thrashosds-health whitelist_health} time: 01:18:34 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965689/ [6965690] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-snappy powercycle/default supported-all-distro/centos_8 tasks/readwrite thrashosds-health whitelist_health} time: 00:18:48 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965690/ [6965691] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zlib powercycle/default supported-all-distro/rhel_8 tasks/snaps-few-objects thrashosds-health whitelist_health} time: 00:28:24 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965691/ [6965692] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-comp-zstd powercycle/default supported-all-distro/ubuntu_latest tasks/snaps-many-objects thrashosds-health whitelist_health} time: 00:25:47 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965692/ [6965693] powercycle/osd/{clusters/3osd-1per-target objectstore/bluestore-hybrid powercycle/default supported-all-distro/centos_8 tasks/admin_socket_objecter_requests thrashosds-health whitelist_health} time: 00:20:51 info: http://pulpito.ceph.com/teuthology-2022-08-10_12:17:02-powercycle-pacific-distro-default-smithi/6965693/