2022-06-02T17:02:17.626 INFO:root:teuthology version: 1.1.1.dev360+g53fc7fef 2022-06-02T17:02:17.627 INFO:teuthology.results:subset: '19/99' 2022-06-02T17:02:17.627 INFO:teuthology.results:seed: '1109' 2022-06-02T17:04:12.753 INFO:root:teuthology version: 1.1.1.dev360+g53fc7fef 2022-06-02T17:04:12.759 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-06-02T18:09:17.672 INFO:teuthology.results:Tests finished! gathering results... 2022-06-02T18:09:18.280 INFO:scrape:Found 10 jobs 2022-06-02T18:09:18.990 INFO:scrape:Found 1 distinct failure reasons 2022-06-02T18:09:18.991 INFO:scrape:Failure: "2022-06-02T17:22:46.531904+0000 mon.a (mon.0) 298 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log 2022-06-02T18:09:18.991 INFO:scrape:2 jobs: ['6860504', '6860503'] 2022-06-02T18:09:18.991 INFO:scrape:suites intersection: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'openstack}', 'rbd-mirror/four-per-cluster', 'rbd:mirror-thrash/{base/install'] 2022-06-02T18:09:18.991 INFO:scrape:suites union: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'openstack}', 'policy/none', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd:mirror-thrash/{base/install', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-fsx-workunit}', 'workloads/rbd-mirror-snapshot-stress-workunit-minimum}'] 2022-06-02T18:09:18.992 INFO:scrape: 2022-06-02T18:09:18.992 INFO:teuthology.results:Sending results to idryomov@gmail.com: Test Run: dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 8 Fail ================================================================= [6860503] rbd:mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum} ----------------------------------------------------------------- time: 00:23:07 info: http://pulpito.ceph.com/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/6860503/ log: http://qa-proxy.ceph.com/teuthology/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/6860503/ sentry: https://sentry.ceph.com/organizations/ceph/?query=48915440ad64459cadc928102534d8e5 "2022-06-02T17:21:26.161783+0000 mon.a (mon.0) 96 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log [6860504] rbd:mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-fsx-workunit} ----------------------------------------------------------------- time: 00:40:21 info: http://pulpito.ceph.com/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/6860504/ log: http://qa-proxy.ceph.com/teuthology/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/6860504/ "2022-06-02T17:22:46.531904+0000 mon.a (mon.0) 298 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log Pass ================================================================= [6860501] rbd:mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} time: 00:54:58 info: http://pulpito.ceph.com/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/6860501/ [6860502] rbd:mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} time: 00:50:23 info: http://pulpito.ceph.com/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/6860502/ [6860505] rbd:mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-stress-workunit} time: 00:59:55 info: http://pulpito.ceph.com/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/6860505/ [6860506] rbd:mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit} time: 00:32:38 info: http://pulpito.ceph.com/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/6860506/ [6860507] rbd:mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} time: 00:51:11 info: http://pulpito.ceph.com/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/6860507/ [6860508] rbd:mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} time: 00:50:01 info: http://pulpito.ceph.com/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/6860508/ [6860509] rbd:mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} time: 00:51:39 info: http://pulpito.ceph.com/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/6860509/ [6860510] rbd:mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum} time: 00:50:13 info: http://pulpito.ceph.com/dis-2022-06-02_17:00:23-rbd:mirror-thrash-wip-dis-testing-distro-default-smithi/6860510/