2022-02-11T10:30:40.158 INFO:root:teuthology version: 1.1.0-7cebb3f2
2022-02-11T10:30:40.159 INFO:teuthology.results:seed: '9720'
2022-02-11T10:56:09.794 INFO:root:teuthology version: 1.1.0-7cebb3f2
2022-02-11T10:56:09.799 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-02-11T12:02:14.751 INFO:teuthology.results:Tests finished! gathering results...
2022-02-11T12:02:19.329 INFO:scrape:Found 54 jobs
2022-02-11T12:02:21.469 INFO:scrape:Found 4 distinct failure reasons
2022-02-11T12:02:21.470 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds
2022-02-11T12:02:21.470 INFO:scrape:1 jobs: ['6676375']
2022-02-11T12:02:21.471 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-comp-zlib', 'openstack}', 'policy/none', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{rhel_8}', 'workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}']
2022-02-11T12:02:21.471 INFO:scrape:
2022-02-11T12:02:21.472 INFO:scrape:Failure: Command failed (workunit test rbd/rbd_mirror_stress.sh) on smithi106 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b73588ff3831b82e6b47d631ee16fc302371d8c6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_IMAGE_FEATURES=layering,exclusive-lock RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh'
2022-02-11T12:02:21.472 INFO:scrape:2 jobs: ['6676388', '6676395']
2022-02-11T12:02:21.473 INFO:scrape:suites intersection: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'openstack}', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}']
2022-02-11T12:02:21.473 INFO:scrape:suites union: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-hybrid', 'openstack}', 'policy/none', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}']
2022-02-11T12:02:21.474 INFO:scrape:
2022-02-11T12:02:21.474 INFO:scrape:Crash: "2022-02-11T11:39:59.256741+0000 mon.a (mon.0) 1711 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log
2022-02-11T12:02:21.475 INFO:scrape:ceph version 17.0.0-10638-ge883dc3b (e883dc3b8274f8cc02cc3a46224cdbf14dd00c74) quincy (dev)
 1: /lib64/libpthread.so.0(+0x12b20) [0x7f33a68cdb20]
 2: gsignal()
 3: abort()
 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1b8) [0x563b0240b60e]
 5: (KernelDevice::_aio_thread()+0xf9e) [0x563b0305547e]
 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x563b03062cb1]
 7: /lib64/libpthread.so.0(+0x814a) [0x7f33a68c314a]
 8: clone()
2022-02-11T12:02:21.475 INFO:scrape:1 jobs: ['6676381']
2022-02-11T12:02:21.476 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'policy/none', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{rhel_8}', 'workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}']
2022-02-11T12:02:21.476 INFO:scrape:
2022-02-11T12:02:21.477 INFO:scrape:Failure: "2022-02-11T10:54:34.268104+0000 mon.a (mon.0) 96 : cluster [WRN] Health check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1 pg degraded (PG_DEGRADED)" in cluster log
2022-02-11T12:02:21.477 INFO:scrape:4 jobs: ['6676371', '6676386', '6676397', '6676391']
2022-02-11T12:02:21.478 INFO:scrape:suites intersection: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'openstack}', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install']
2022-02-11T12:02:21.478 INFO:scrape:suites union: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-stupid', 'openstack}', 'policy/none', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}', 'workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}', 'workloads/rbd-mirror-snapshot-stress-workunit-minimum}']
2022-02-11T12:02:21.479 INFO:scrape:
2022-02-11T12:02:21.482 INFO:teuthology.results:Sending results to to.my.trociny@gmail.com: Test Run: trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/
failed:  8
dead:    0
running: 0
waiting: 0
queued:  0
passed:  46


Fail
=================================================================
[6676371]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
-----------------------------------------------------------------
time:   00:15:30
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676371/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676371/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f97caad369ff4a7baa38cfec76fe1770

    "2022-02-11T10:54:34.268104+0000 mon.a (mon.0) 96 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6676375]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
-----------------------------------------------------------------
time:   00:29:10
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676375/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676375/
sentry: https://sentry.ceph.com/organizations/ceph/?query=55a6d6b496d24fe6962f0cdedfeca7e7

    reached maximum tries (90) after waiting for 540 seconds

[6676381]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
-----------------------------------------------------------------
time:   01:05:37
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676381/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676381/

    "2022-02-11T11:39:59.256741+0000 mon.a (mon.0) 1711 : cluster [WRN] Health
    check failed: 1 osds down (OSD_DOWN)" in cluster log

[6676386]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
-----------------------------------------------------------------
time:   00:48:33
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676386/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676386/

    "2022-02-11T11:00:10.070702+0000 mon.a (mon.0) 202 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg inactive, 1 pg peering
    (PG_AVAILABILITY)" in cluster log

[6676388]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
-----------------------------------------------------------------
time:   00:10:49
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676388/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676388/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9d3e4aacbc5248b68e20f151c24d31e8

    Command failed (workunit test rbd/rbd_mirror_stress.sh) on smithi106 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=b73588ff3831b82e6b47d631ee16fc302371d8c6
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image
    RBD_IMAGE_FEATURES=layering,exclusive-lock RBD_MIRROR_INSTANCES=4
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-
    ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /ho
    me/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror
    _stress.sh'

[6676391]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
-----------------------------------------------------------------
time:   00:48:51
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676391/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676391/

    "2022-02-11T11:03:01.014621+0000 mon.a (mon.0) 195 : cluster [WRN] Health
    check failed: Degraded data redundancy: 3/334 objects degraded (0.898%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6676395]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
-----------------------------------------------------------------
time:   00:10:20
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676395/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676395/
sentry: https://sentry.ceph.com/organizations/ceph/?query=af112dd3958f40a38d2841ac6bdf80b2

    Command failed (workunit test rbd/rbd_mirror_stress.sh) on smithi181 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=b73588ff3831b82e6b47d631ee16fc302371d8c6
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image
    RBD_IMAGE_FEATURES=layering,exclusive-lock RBD_MIRROR_INSTANCES=4
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-
    ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /ho
    me/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror
    _stress.sh'

[6676397]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
-----------------------------------------------------------------
time:   00:44:57
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676397/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676397/

    "2022-02-11T11:06:55.428493+0000 mon.a (mon.0) 203 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg inactive, 1 pg peering
    (PG_AVAILABILITY)" in cluster log



Pass
=================================================================
[6676348] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:51:40
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676348/

[6676349] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:51:14
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676349/

[6676350] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:50
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676350/

[6676351] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:46:24
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676351/

[6676352] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   01:05:18
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676352/

[6676353] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:50
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676353/

[6676354] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:47:01
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676354/

[6676355] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   01:01:51
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676355/

[6676356] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:48:13
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676356/

[6676357] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:10
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676357/

[6676358] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   01:00:02
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676358/

[6676359] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:50:49
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676359/

[6676360] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:58
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676360/

[6676361] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:46:52
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676361/

[6676362] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:52:20
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676362/

[6676363] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:16
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676363/

[6676364] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:45:06
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676364/

[6676365] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:51:09
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676365/

[6676366] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:49:05
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676366/

[6676367] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:48:34
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676367/

[6676368] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:46:10
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676368/

[6676369] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:50:34
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676369/

[6676370] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:59:12
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676370/

[6676372] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:49:36
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676372/

[6676373] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:46:57
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676373/

[6676374] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:48:50
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676374/

[6676376] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:51:53
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676376/

[6676377] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:05
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676377/

[6676378] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:46:13
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676378/

[6676379] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:51:30
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676379/

[6676380] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:58:58
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676380/

[6676382] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:50:45
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676382/

[6676383] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:59:48
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676383/

[6676384] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:48:03
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676384/

[6676385] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:46:20
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676385/

[6676387] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:46:37
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676387/

[6676389] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:48:02
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676389/

[6676390] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:17
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676390/

[6676392] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:48:10
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676392/

[6676393] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:52:04
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676393/

[6676394] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:59:50
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676394/

[6676396] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   01:06:32
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676396/

[6676398] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   01:01:21
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676398/

[6676399] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:50:00
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676399/

[6676400] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:48:23
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676400/

[6676401] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:46:00
info:   http://pulpito.ceph.com/trociny-2022-02-11_10:28:15-rbd-master-distro-basic-smithi/6676401/