Found 7 jobs Found 3 distinct failure reasons Dead: b"2022-12-05T23:50:56.556 INFO:tasks.workunit.cluster1.client.mirror.smithi182.stderr:+++ rbd --cluster cluster1 -p mirror --namespace '' ls" ceph version 18.0.0-1168-g1de1c016 (1de1c01608678ccbe5faeb1a4eaff28f84bea777) reef (dev) 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fc5bd8a2cf0] 2: gsignal() 3: abort() 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x55a29385e659] 5: (KernelDevice::_aio_thread()+0x1798) [0x55a2944f80f8] 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x55a2944fa481] 7: /lib64/libpthread.so.0(+0x81ca) [0x7fc5bd8981ca] 8: clone() 1 jobs: ['7103355'] suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-stupid', 'openstack}', 'policy/none', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{centos_8}', 'workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}'] Failure: "1670245078.2540357 osd.0 (osd.0) 3 : cluster [WRN] 64 slow requests (by type [ 'delayed' : 26 'waiting for sub ops' : 38 ] most affected pool [ 'cache' : 64 ])" in cluster log 1 jobs: ['7103353'] suites: ['clusters/{fixed-3', 'features/readbalance', 'msgr-failures/few', 'objectstore/filestore-xfs', 'openstack}', 'pool/ec-cache-pool', 'rbd/qemu/{cache/writeback', 'supported-random-distro$/{ubuntu_latest}', 'workloads/qemu_bonnie}'] Failure: "1670244772.975793 mon.a (mon.0) 200 : cluster [WRN] Health check failed: Degraded data redundancy: 1/2080 objects degraded (0.048%), 1 pg degraded (PG_DEGRADED)" in cluster log 1 jobs: ['7103354'] suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-comp-zstd', 'openstack}', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{rhel_8}', 'workloads/rbd-mirror-fsx-workunit}']