2023-04-02T14:50:32.701 INFO:root:teuthology version: 0.0.1.dev91+g5338cb9 2023-04-02T14:50:32.702 INFO:teuthology.results:subset: '111/12000' 2023-04-02T14:50:32.703 INFO:teuthology.results:seed: '559' 2023-04-02T14:52:44.567 INFO:root:teuthology version: 0.0.1.dev91+g5338cb9 2023-04-02T14:52:44.571 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-04-03T02:53:34.229 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2023-04-03T02:53:35.204 INFO:scrape:Found 14 jobs 2023-04-03T02:53:36.715 INFO:scrape:Found 3 distinct failure reasons 2023-04-03T02:53:36.716 INFO:scrape:Failure: "1680448419.0240686 mon.a (mon.0) 186 : cluster [WRN] Health check failed: Degraded data redundancy: 1/22 objects degraded (4.545%), 1 pg degraded (PG_DEGRADED)" in cluster log 2023-04-03T02:53:36.716 INFO:scrape:1 jobs: ['7230049'] 2023-04-03T02:53:36.716 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'policy/none', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-fsx-workunit}'] 2023-04-03T02:53:36.717 INFO:scrape: 2023-04-03T02:53:36.717 INFO:scrape:Failure: Command failed on smithi173 with status 1: 'test -f /home/ubuntu/cephtest/archive/qemu/client.0/success' 2023-04-03T02:53:36.717 INFO:scrape:1 jobs: ['7230053'] 2023-04-03T02:53:36.718 INFO:scrape:suites: ['2-cluster/{fix-2', '3-supported-random-distro$/{ubuntu_latest}', '4-cache-path', '5-cache-mode/rwl', '6-cache-size/1G', '7-workloads/qemu_xfstests}', 'openstack}', 'rbd/pwl-cache/tmpfs/{1-base/install'] 2023-04-03T02:53:36.718 INFO:scrape: 2023-04-03T02:53:36.718 INFO:scrape:Dead: b'2023-04-03T02:53:35.730 INFO:tasks.ceph.osd.2.smithi033.stderr:2023-04-03T02:53:35.732+0000 7f4e99f20700 -1 osd.2 24 get_health_metrics reporting 98 slow ops, oldest is osd_op(osd.1.9:11400 2.0 2:10212668:::rbd_data.10769fbb89b6.0000000000000024:head [copy-from ver 12540 in=76b] snapc 4=[4] ondisk+write+ignore_overlay+enforce_snapc+known_if_redirected+supports_pool_eio e24)' 2023-04-03T02:53:36.719 INFO:scrape:1 jobs: ['7230048'] 2023-04-03T02:53:36.719 INFO:scrape:suites: ['clusters/{fixed-1', 'features/layering', 'msgr-failures/few', 'objectstore/bluestore-comp-zstd', 'openstack}', 'pool/small-cache-pool', 'rbd/cli/{base/install', 'supported-random-distro$/{centos_8}', 'workloads/rbd_cli_migration}'] 2023-04-03T02:53:36.719 INFO:scrape: