Found 172 jobs Found 6 distinct failure reasons Timeout 3h running clone.client.0/qa/workunits/rbd/qemu_dynamic_features.sh 3 jobs: ['5881341', '5881445', '5881433'] suites intersection: ['clusters/{fixed-3', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{ubuntu_latest}'] suites union: ['clusters/{fixed-3', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/dynamic_features_no_cache}', 'workloads/dynamic_features}'] Timeout 3h running clone.client.0/qa/workunits/rbd/qemu_rebuild_object_map.sh 1 jobs: ['5881456'] suites: ['clusters/{fixed-3', 'objectstore/bluestore-stupid', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rebuild_object_map}'] Failure: Command failed on gibba001 with status 123: "sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --" 1 jobs: ['5881330'] suites: ['clusters/{fixed-3', 'features/defaults', 'msgr-failures/few', 'objectstore/bluestore-comp-zstd', 'openstack}', 'pool/ec-data-pool', 'rbd/encryption/{cache/writethrough', 'supported-random-distro$/{rhel_8}', 'workloads/qemu_xfstests_luks1}'] Dead: b'2021-02-16T05:20:02.251 DEBUG:teuthology.exit:Finished running handlers' 1 jobs: ['5881310'] suites: ['cluster/{fixed-3', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'openstack}', 'rbd/nbd/{base/install', 'supported-random-distro$/{rhel_8}', 'thrashers/cache', 'thrashosds-health', 'workloads/rbd_fsx_nbd}'] Crash: "2021-02-16T22:18:18.602481+0000 mon.a (mon.0) 188 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log ceph version 17.0.0-762-g8c9cf9f7 (8c9cf9f7b1967d7b3c7efa1f243ae870fd5ac11a) quincy (dev) 1: /lib64/libpthread.so.0(+0x12b20) [0x7fbf89450b20] 2: gsignal() 3: abort() 4: /lib64/libstdc++.so.6(+0x9009b) [0x7fbf88a6e09b] 5: /lib64/libstdc++.so.6(+0x9653c) [0x7fbf88a7453c] 6: /lib64/libstdc++.so.6(+0x95559) [0x7fbf88a73559] 7: __gxx_personality_v0() 8: /lib64/libgcc_s.so.1(+0x10b13) [0x7fbf88454b13] 9: _Unwind_Resume() 10: ceph-osd(+0x56712e) [0x555d2d8a212e] 11: (ShardedThreadPool::WorkThreadSharded::entry()+0x14) [0x555d2e02f4e4] 12: (Thread::_entry_func(void*)+0xd) [0x555d2e01e34d] 13: /lib64/libpthread.so.0(+0x814a) [0x7fbf8944614a] 14: clone() 1 jobs: ['5881390'] suites: ['clusters/{fixed-3', 'objectstore/bluestore-comp-snappy', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{rhel_8}', 'workloads/dynamic_features}'] Timeout 6h running clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 1 jobs: ['5881377'] suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}']