2022-08-14T17:50:26.483 INFO:root:teuthology version: 1.1.1.dev409+g9e7483cc 2022-08-14T17:50:26.485 INFO:teuthology.results:subset: '91/99' 2022-08-14T17:50:26.485 INFO:teuthology.results:seed: '5599' 2022-08-14T18:23:48.585 INFO:root:teuthology version: 1.1.1.dev409+g9e7483cc 2022-08-14T18:23:48.588 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-08-14T20:01:55.880 INFO:teuthology.results:Tests finished! gathering results... 2022-08-14T20:01:57.450 INFO:scrape:Found 19 jobs 2022-08-14T20:01:58.311 INFO:scrape:Found 2 distinct failure reasons 2022-08-14T20:01:58.312 INFO:scrape:Failure: "2022-08-14T18:10:03.956829+0000 mon.a (mon.0) 213 : cluster [WRN] Health check failed: Degraded data redundancy: 2/2118 objects degraded (0.094%), 1 pg degraded (PG_DEGRADED)" in cluster log 2022-08-14T20:01:58.313 INFO:scrape:1 jobs: ['6972493'] 2022-08-14T20:01:58.313 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-comp-lz4', 'openstack}', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{centos_8}', 'workloads/rbd-mirror-fsx-workunit}'] 2022-08-14T20:01:58.314 INFO:scrape: 2022-08-14T20:01:58.314 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds 2022-08-14T20:01:58.315 INFO:scrape:1 jobs: ['6972500'] 2022-08-14T20:01:58.315 INFO:scrape:suites: ['clusters/{fixed-3', 'config/copy-on-read', 'min-compat-client/default', 'msgr-failures/few', 'objectstore/bluestore-comp-snappy', 'openstack}', 'pool/none', 'rbd/librbd/{cache/writethrough', 'supported-random-distro$/{centos_8}', 'workloads/python_api_tests_with_defaults}'] 2022-08-14T20:01:58.315 INFO:scrape: 2022-08-14T20:01:58.317 INFO:teuthology.results:Sending results to idryomov@gmail.com: Test Run: dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 17 Fail ================================================================= [6972493] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit} ----------------------------------------------------------------- time: 00:37:09 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972493/ log: http://qa-proxy.ceph.com/teuthology/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972493/ "2022-08-14T18:10:03.956829+0000 mon.a (mon.0) 213 : cluster [WRN] Health check failed: Degraded data redundancy: 2/2118 objects degraded (0.094%), 1 pg degraded (PG_DEGRADED)" in cluster log [6972500] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/default msgr-failures/few objectstore/bluestore-comp-snappy pool/none supported-random-distro$/{centos_8} workloads/python_api_tests_with_defaults} ----------------------------------------------------------------- time: 00:17:23 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972500/ log: http://qa-proxy.ceph.com/teuthology/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972500/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9c13f8f8ff3d41efb3e6737de6f19220 reached maximum tries (90) after waiting for 540 seconds Pass ================================================================= [6972491] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-snappy pool/replicated-data-pool supported-random-distro$/{ubuntu_latest} workloads/c_api_tests_with_journaling} time: 00:32:14 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972491/ [6972492] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/none min-compat-client/octopus msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/replicated-data-pool supported-random-distro$/{ubuntu_latest} workloads/python_api_tests_with_journaling} time: 00:18:46 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972492/ [6972494] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{ubuntu_latest} workloads/rbd_fio} time: 00:26:16 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972494/ [6972495] rbd/pwl-cache/tmpfs/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{rhel_8} 4-cache-path 5-cache-mode/rwl 6-cache-size/5G 7-workloads/qemu_xfstests} time: 01:07:23 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972495/ [6972496] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/none min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-zlib pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/python_api_tests} time: 00:24:19 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972496/ [6972497] rbd/encryption/{cache/writearound clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-comp-zlib pool/replicated-data-pool supported-random-distro$/{centos_8} workloads/qemu_xfstests_luks1} time: 01:15:34 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972497/ [6972498] rbd/encryption/{cache/writeback clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-comp-zstd pool/small-cache-pool supported-random-distro$/{rhel_8} workloads/qemu_xfstests_luks2} time: 01:07:39 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972498/ [6972499] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/none min-compat-client/default msgr-failures/few objectstore/bluestore-bitmap pool/small-cache-pool supported-random-distro$/{centos_8} workloads/fsx} time: 00:15:12 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972499/ [6972501] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-comp-lz4 pool/none supported-random-distro$/{ubuntu_latest} workloads/python_api_tests_with_defaults} time: 00:17:59 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972501/ [6972502] rbd/maintenance/{base/install clusters/{fixed-3 openstack} objectstore/bluestore-comp-zlib qemu/xfstests supported-random-distro$/{centos_8} workloads/dynamic_features} time: 01:42:18 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972502/ [6972503] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-zstd pool/ec-data-pool supported-random-distro$/{centos_8} workloads/c_api_tests} time: 00:23:01 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972503/ [6972504] rbd/pwl-cache/tmpfs/{1-base/install 2-cluster/{fix-2 openstack} 3-supported-random-distro$/{rhel_8} 4-cache-path 5-cache-mode/ssd 6-cache-size/5G 7-workloads/qemu_xfstests} time: 01:02:45 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972504/ [6972505] rbd/basic/{base/install cachepool/none clusters/{fixed-1 openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} tasks/rbd_cls_tests} time: 00:07:30 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972505/ [6972506] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/octopus msgr-failures/few objectstore/bluestore-hybrid pool/replicated-data-pool supported-random-distro$/{centos_8} workloads/c_api_tests_with_journaling} time: 00:30:15 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972506/ [6972507] rbd/cli/{base/install clusters/{fixed-1 openstack} features/layering msgr-failures/few objectstore/bluestore-stupid pool/none supported-random-distro$/{ubuntu_latest} workloads/rbd_cli_luks_encryption} time: 00:06:40 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972507/ [6972508] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/none min-compat-client/octopus msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/python_api_tests} time: 00:23:52 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972508/ [6972509] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/octopus msgr-failures/few objectstore/filestore-xfs pool/replicated-data-pool supported-random-distro$/{centos_8} workloads/python_api_tests_with_journaling} time: 00:18:33 info: http://pulpito.ceph.com/dis-2022-08-14_17:49:24-rbd-wip-dis-testing-2-distro-default-smithi/6972509/