2021-08-23T13:40:31.477 INFO:root:teuthology version: 1.1.0-321319b1
2021-08-23T13:40:31.478 INFO:teuthology.results:subset: '3/10'
2021-08-23T13:40:31.479 INFO:teuthology.results:seed: '9918'
2021-08-23T13:53:53.240 INFO:root:teuthology version: 1.1.0-321319b1
2021-08-23T13:53:53.245 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-08-23T14:57:58.066 INFO:teuthology.results:Tests finished! gathering results...
2021-08-23T14:58:07.208 INFO:scrape:Found 30 jobs
2021-08-23T14:58:07.786 INFO:scrape:Found 1 distinct failure reasons
2021-08-23T14:58:07.787 INFO:scrape:Failure: "2021-08-23T14:05:06.892628+0000 mon.a (mon.0) 113 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log
2021-08-23T14:58:07.787 INFO:scrape:3 jobs: ['6352688', '6352692', '6352700']
2021-08-23T14:58:07.788 INFO:scrape:suites intersection: ['cluster/{2-node', 'msgr-failures/few', 'openstack}']
2021-08-23T14:58:07.788 INFO:scrape:suites union: ['clients/mirror', 'clients/{mirror-extra', 'cluster/{2-node', 'mirror}', 'msgr-failures/few', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/filestore-xfs', 'openstack}', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'rbd/mirror/{base/install', 'supported-random-distro$/{centos_8.stream}', 'supported-random-distro$/{rhel_8}', 'workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}', 'workloads/rbd-mirror-snapshot-workunit-fast-diff}', 'workloads/rbd-mirror-workunit-config-key}']
2021-08-23T14:58:07.788 INFO:scrape:
2021-08-23T14:58:07.790 INFO:teuthology.results:Sending results to to.my.trociny@gmail.com: Test Run: trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/
failed:  3
dead:    0
running: 0
waiting: 0
queued:  0
passed:  27


Fail
=================================================================
[6352688]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{rhel_8} workloads/rbd-mirror-workunit-config-key}
-----------------------------------------------------------------
time:   00:26:52
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352688/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352688/
sentry: https://sentry.ceph.com/organizations/ceph/?query=69cfb0404a714515b24adb9d95d37483

    "2021-08-23T14:05:06.892628+0000 mon.a (mon.0) 113 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6352692]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
-----------------------------------------------------------------
time:   00:18:17
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352692/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352692/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4e93dbae33c845c09b95cf0dfec8cb8f

    "2021-08-23T14:04:37.457141+0000 mon.a (mon.0) 109 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6352700]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-snapshot-workunit-fast-diff}
-----------------------------------------------------------------
time:   00:19:32
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352700/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352700/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8bffc2d3fb0e45c7829eb9ccc0359468

    "2021-08-23T14:06:45.409273+0000 mon.a (mon.0) 88 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log



Pass
=================================================================
[6352684] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-workunit-minimum}
time:   00:32:04
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352684/

[6352685] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-workunit}
time:   00:33:14
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352685/

[6352686] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus}
time:   00:53:30
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352686/

[6352687] rbd/singleton/{all/rbd_mirror objectstore/bluestore-hybrid openstack supported-random-distro$/{ubuntu_latest}}
time:   00:41:51
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352687/

[6352689] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:48:33
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352689/

[6352690] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{rhel_8} workloads/rbd-mirror-workunit-min-compat-client-octopus}
time:   00:32:54
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352690/

[6352691] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-policy-none}
time:   00:23:09
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352691/

[6352693] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-snappy validator/memcheck workloads/rbd_mirror}
time:   00:56:12
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352693/

[6352694] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-workunit-policy-simple}
time:   00:24:29
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352694/

[6352695] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} workloads/rbd-mirror-bootstrap-workunit}
time:   00:11:22
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352695/

[6352696] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:50:29
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352696/

[6352697] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-ha-workunit}
time:   00:18:10
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352697/

[6352698] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   01:01:58
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352698/

[6352699] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-snapshot-workunit-exclusive-lock}
time:   00:22:37
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352699/

[6352701] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit}
time:   00:40:35
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352701/

[6352702] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-workunit-journaling}
time:   00:33:05
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352702/

[6352703] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-workunit-minimum}
time:   00:31:43
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352703/

[6352704] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-journal-stress-workunit}
time:   00:57:50
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352704/

[6352705] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus}
time:   00:54:39
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352705/

[6352706] rbd/singleton/{all/rbd_mirror objectstore/bluestore-comp-zstd openstack supported-random-distro$/{centos_8.stream}}
time:   00:44:13
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352706/

[6352707] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} workloads/rbd-mirror-workunit-config-key}
time:   00:33:57
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352707/

[6352708] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-journal-workunit}
time:   00:33:53
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352708/

[6352709] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-workunit-min-compat-client-octopus}
time:   00:23:48
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352709/

[6352710] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-workunit-policy-none}
time:   00:23:44
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352710/

[6352711] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:57:18
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352711/

[6352712] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-lz4 validator/memcheck workloads/rbd_mirror}
time:   00:52:49
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352712/

[6352713] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8.stream} workloads/rbd-mirror-workunit-policy-simple}
time:   00:24:14
info:   http://pulpito.ceph.com/trociny-2021-08-23_13:16:42-rbd-wip-mgolub-testing-distro-basic-smithi/6352713/