2023-07-29T20:08:29.099 INFO:root:teuthology version: 0.0.1.dev172+g6114016
2023-07-29T20:08:29.100 INFO:teuthology.results:subset: '111/120000'
2023-07-29T20:08:29.100 INFO:teuthology.results:seed: '9675'
2023-07-29T20:08:33.554 INFO:root:teuthology version: 0.0.1.dev172+g6114016
2023-07-29T20:08:33.555 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-07-30T08:09:28.880 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds
2023-07-30T08:09:29.850 INFO:scrape:Found 10 jobs
2023-07-30T08:09:33.629 INFO:scrape:Found 5 distinct failure reasons
2023-07-30T08:09:33.630 INFO:scrape:Crash: "2023-07-29T21:13:42.396455+0000 mon.a (mon.0) 1500 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log
2023-07-30T08:09:33.630 INFO:scrape:ceph version 18.1.3-11-g0484097b (0484097b0572e96f42cf6402cbe0ac8dcb046577) reef (rc)
 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f2822d18520]
 2: pthread_kill()
 3: raise()
 4: abort()
 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x17b) [0x56442df8ed55]
 6: (KernelDevice::_aio_thread()+0x9bc) [0x56442e845bac]
 7: ceph-osd(+0x1576461) [0x56442e846461]
 8: /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f2822d6ab43]
 9: /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f2822dfca00]
2023-07-30T08:09:33.631 INFO:scrape:1 jobs: ['7357465']
2023-07-30T08:09:33.631 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-stupid', 'openstack}', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}']
2023-07-30T08:09:33.631 INFO:scrape:
2023-07-30T08:09:33.632 INFO:scrape:Failure: "2023-07-29T20:44:47.841187+0000 mon.a (mon.0) 183 : cluster [WRN] Health check failed: Degraded data redundancy: 1/22 objects degraded (4.545%), 1 pg degraded (PG_DEGRADED)" in cluster log
2023-07-30T08:09:33.632 INFO:scrape:1 jobs: ['7357456']
2023-07-30T08:09:33.632 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'policy/none', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-fsx-workunit}']
2023-07-30T08:09:33.633 INFO:scrape:
2023-07-30T08:09:33.633 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/diff_continuous.sh
2023-07-30T08:09:33.633 INFO:scrape:1 jobs: ['7357460']
2023-07-30T08:09:33.634 INFO:scrape:suites: ['cluster/{fixed-3', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'openstack}', 'rbd/nbd/{base/install', 'supported-random-distro$/{centos_latest}', 'thrashers/cache', 'thrashosds-health', 'workloads/rbd_nbd_diff_continuous}']
2023-07-30T08:09:33.634 INFO:scrape:
2023-07-30T08:09:33.634 INFO:scrape:Dead: b'2023-07-30T08:09:30.953 INFO:tasks.ceph.osd.2.smithi118.stderr:2023-07-30T08:09:30.902+0000 7f30f2176640 -1 osd.2 26 get_health_metrics reporting 144 slow ops, oldest is osd_op(osd.0.8:957 2.1 2:9cff77d9:::rbd_data.107d5bec0aaa.0000000000000020:head [copy-from ver 513 in=76b] snapc 0=[] ondisk+write+ignore_overlay+enforce_snapc+known_if_redirected+supports_pool_eio e25)'
2023-07-30T08:09:33.634 INFO:scrape:1 jobs: ['7357457']
2023-07-30T08:09:33.635 INFO:scrape:suites: ['clusters/{fixed-1', 'features/layering', 'msgr-failures/few', 'objectstore/bluestore-comp-zstd', 'openstack}', 'pool/small-cache-pool', 'rbd/cli/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd_cli_migration}']
2023-07-30T08:09:33.635 INFO:scrape:
2023-07-30T08:09:33.635 INFO:scrape:Failure: ['MODULE FAILURE', 'See stdout/stderr for the exact error']
2023-07-30T08:09:33.636 INFO:scrape:1 jobs: ['7357464']
2023-07-30T08:09:33.636 INFO:scrape:suites: ['cluster/{fixed-3', 'msgr-failures/few', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rbd/nbd/{base/install', 'supported-random-distro$/{ubuntu_20.04}', 'thrashers/cache', 'thrashosds-health', 'workloads/rbd_nbd_diff_continuous}']
2023-07-30T08:09:33.636 INFO:scrape: