Found 181 jobs Found 8 distinct failure reasons Failure: "2022-07-06T05:05:39.086251+0000 mon.a (mon.0) 297 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,b (MON_DOWN)" in cluster log 1 jobs: ['6915480'] suites: ['clusters/{fixed-3', 'features/defaults', 'msgr-failures/few', 'objectstore/filestore-xfs', 'openstack}', 'pool/ec-cache-pool', 'rbd/qemu/{cache/writethrough', 'supported-random-distro$/{centos_8}', 'workloads/qemu_xfstests}'] Timeout 3h running clone.client.0/qa/workunits/rbd/qemu_rebuild_object_map.sh 1 jobs: ['6915375'] suites: ['clusters/{fixed-3', 'objectstore/bluestore-comp-zlib', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rebuild_object_map}'] Dead: b'2022-07-06T14:52:21.506 DEBUG:teuthology.exit:Finished running handlers' ceph version 16.2.9-446-g2103eaf0 (2103eaf02dd3c9da5290a33a5b7ec7f42138d76a) pacific (stable) 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f45e9e3ace0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x7f45ec100bb9] 5: /usr/lib64/ceph/libceph-common.so.2(+0x277d82) [0x7f45ec100d82] 6: (Monitor::~Monitor()+0xe40) [0x555f46c1c3a0] 7: (Monitor::~Monitor()+0xd) [0x555f46c1c3fd] 8: main() 9: __libc_start_main() 10: _start() 1 jobs: ['6915373'] suites: ['2-cluster/{fix-2', '3-supported-random-distro$/{centos_8}', '4-cache-path', '5-cache-mode/rwl', '6-cache-size/5G', '7-workloads/qemu_xfstests}', 'openstack}', 'rbd/pwl-cache/tmpfs/{1-base/install'] Failure: Command failed on smithi081 with status 1: 'test -f /home/ubuntu/cephtest/archive/qemu/client.0/success' 2 jobs: ['6915509', '6915434'] suites intersection: ['openstack}'] suites union: ['2-cluster/{fix-2', '2-clusters/{fixed-3', '3-objectstore/bluestore-comp-lz4', '3-supported-random-distro$/{rhel_8}', '4-cache-path', '4-supported-random-distro$/{ubuntu_latest}', '5-cache-mode/ssd', '5-pool/replicated-data-pool', '6-cache-size/1G', '6-prepare/raw-file', '7-io-workloads/qemu_xfstests', '7-workloads/qemu_xfstests}', '8-migrate-workloads/execute', '9-cleanup/cleanup}', 'openstack}', 'rbd/migration/{1-base/install', 'rbd/pwl-cache/tmpfs/{1-base/install'] Timeout 3h running clone.client.0/qa/workunits/rbd/qemu_dynamic_features.sh 3 jobs: ['6915404', '6915495', '6915457'] suites intersection: ['clusters/{fixed-3', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/dynamic_features_no_cache}'] suites union: ['clusters/{fixed-3', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-hybrid', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/dynamic_features_no_cache}'] Dead: b'2022-07-06T15:23:58.436 DEBUG:teuthology.exit:Finished running handlers' 1 jobs: ['6915426'] suites: ['clusters/{fixed-3', 'features/defaults', 'msgr-failures/few', 'objectstore/bluestore-hybrid', 'openstack}', 'pool/none', 'rbd/qemu/{cache/none', 'supported-random-distro$/{ubuntu_latest}', 'workloads/qemu_xfstests}'] Failure: {'smithi050.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}} 2 jobs: ['6915396', '6915419'] suites intersection: ['msgr-failures/few', 'openstack}', 'supported-random-distro$/{rhel_8}', 'thrashosds-health'] suites union: ['cluster/{fixed-3', 'clusters/{fixed-2', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-hybrid', 'openstack}', 'rbd/nbd/{base/install', 'rbd/thrash/{base/install', 'supported-random-distro$/{rhel_8}', 'thrashers/cache', 'thrashers/default', 'thrashosds-health', 'workloads/journal}', 'workloads/rbd_fsx_nbd}'] Timeout 3h running clone.client.0/qa/workunits/rbd/cli_generic.sh 1 jobs: ['6915479'] suites: ['clusters/{fixed-1', 'features/layering', 'msgr-failures/few', 'objectstore/filestore-xfs', 'openstack}', 'pool/small-cache-pool', 'rbd/cli/{base/install', 'supported-random-distro$/{centos_8}', 'workloads/rbd_cli_generic}']