2022-04-08T23:29:46.940 INFO:root:teuthology version: 1.1.0-a347b2ac
2022-04-08T23:29:46.941 INFO:teuthology.results:seed: '8044'
2022-04-09T05:53:32.767 INFO:root:teuthology version: 1.1.0-a347b2ac
2022-04-09T05:53:32.820 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-04-09T06:51:37.593 INFO:teuthology.results:Tests finished! gathering results...
2022-04-09T06:51:52.908 INFO:scrape:Found 190 jobs
2022-04-09T06:51:57.523 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783173/teuthology.log
2022-04-09T06:52:05.031 INFO:scrape:Found 8 distinct failure reasons
2022-04-09T06:52:05.032 INFO:scrape:Failure: Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi178 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=69ad68d989715754945f23c59c7f3abe9a8b52dc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 USE_CLUSTER3=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh'
2022-04-09T06:52:05.033 INFO:scrape:5 jobs: ['6783112', '6783163', '6783082', '6783020', '6783196']
2022-04-09T06:52:05.033 INFO:scrape:suites intersection: ['cluster/3-nodes/{base/{install', 'extra/{rbd-mirror}', 'mirror', 'msgr-failures/few', 'rbd/mirror-thrash/{base/{openstack}', 'roles}', 'supported-random-distro$/{centos_8}}', 'workloads/rbd-mirror-journal-workunit-multipeer}']
2022-04-09T06:52:05.034 INFO:scrape:suites union: ['cluster/3-nodes/{base/{install', 'extra/{rbd-mirror}', 'mirror', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-stupid', 'policy/none', 'policy/simple', 'rbd/mirror-thrash/{base/{openstack}', 'roles}', 'supported-random-distro$/{centos_8}}', 'workloads/rbd-mirror-journal-workunit-multipeer}']
2022-04-09T06:52:05.034 INFO:scrape:
2022-04-09T06:52:05.035 INFO:scrape:Crash: Found coredumps on ubuntu@smithi081.front.sepia.ceph.com
2022-04-09T06:52:05.035 INFO:scrape:ceph version 17.0.0-11492-g69ad68d9 (69ad68d989715754945f23c59c7f3abe9a8b52dc) quincy (dev)
 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x143c0) [0x7f446f5f43c0]
 2: gsignal()
 3: abort()
 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b7) [0x7f446fb56834]
 5: /usr/lib/ceph/libceph-common.so.2(+0x27b9de) [0x7f446fb569de]
 6: (rbd::mirror::NamespaceReplayer<librbd::ImageCtx>::add_peer(rbd::mirror::Peer<librbd::ImageCtx> const&, Context*)+0x3db) [0x55cf5376594b]
 7: (rbd::mirror::PoolReplayer<librbd::ImageCtx>::update_namespace_replayers()+0x6e5) [0x55cf53706aa5]
 8: (rbd::mirror::PoolReplayer<librbd::ImageCtx>::run()+0xa85) [0x55cf537081e5]
 9: (rbd::mirror::PoolReplayer<librbd::ImageCtx>::PoolReplayerThread::entry()+0x11) [0x55cf53708e11]
 10: /lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f446f5e8609]
 11: clone()
2022-04-09T06:52:05.035 INFO:scrape:1 jobs: ['6783153']
2022-04-09T06:52:05.036 INFO:scrape:suites: ['cluster/3-nodes/{base/{install', 'extra/{rbd-mirror}', 'mirror', 'msgr-failures/few', 'objectstore/bluestore-comp-snappy', 'policy/none', 'rbd/mirror-thrash/{base/{openstack}', 'roles}', 'supported-random-distro$/{ubuntu_latest}}', 'workloads/rbd-mirror-journal-workunit-multipeer}']
2022-04-09T06:52:05.036 INFO:scrape:
2022-04-09T06:52:05.037 INFO:scrape:Failure: "2022-04-09T05:47:23.758564+0000 mon.a (mon.0) 1759 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log
2022-04-09T06:52:05.037 INFO:scrape:12 jobs: ['6783159', '6783101', '6783019', '6783126', '6783106', '6783088', '6783060', '6783038', '6783184', '6783090', '6783110', '6783151']
2022-04-09T06:52:05.038 INFO:scrape:suites intersection: []
2022-04-09T06:52:05.038 INFO:scrape:suites union: ['centos_latest', 'cluster/2-nodes/{base/{install', 'clusters/{fixed-1', 'extra/{mirror-extra}', 'extra/{rbd-mirror}', 'mirror', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'openstack}', 'policy/none', 'policy/simple', 'rbd/mirror-thrash/{base/{openstack}', 'rbd/mirror/{base/{openstack}', 'rbd/valgrind/{base/install', 'roles}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{ubuntu_latest}}', 'validator/memcheck', 'workloads/rbd-mirror-fsx-workunit}', 'workloads/rbd-mirror-snapshot-bootstrap-workunit}', 'workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}', 'workloads/rbd-mirror-snapshot-stress-workunit-minimum}', 'workloads/rbd_mirror}']
2022-04-09T06:52:05.039 INFO:scrape:
2022-04-09T06:52:05.039 INFO:scrape:Failure: Found coredumps on ubuntu@smithi036.front.sepia.ceph.com
2022-04-09T06:52:05.040 INFO:scrape:1 jobs: ['6783084']
2022-04-09T06:52:05.040 INFO:scrape:suites: ['cluster/2-nodes/{base/{install', 'extra/{mirror-extra}', 'mirror', 'msgr-failures/few', 'objectstore/bluestore-low-osd-mem-target', 'rbd/mirror/{base/{openstack}', 'roles}', 'supported-random-distro$/{centos_8}}', 'workloads/rbd-mirror-snapshot-bootstrap-workunit}']
2022-04-09T06:52:05.041 INFO:scrape:
2022-04-09T06:52:05.041 INFO:scrape:Crash: "2022-04-09T01:19:03.689900+0000 mon.a (mon.0) 1458 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log
2022-04-09T06:52:05.042 INFO:scrape:ceph version 17.0.0-11492-g69ad68d9 (69ad68d989715754945f23c59c7f3abe9a8b52dc) quincy (dev)
 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f27915c5ce0]
 2: gsignal()
 3: abort()
 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1b8) [0x560cacfc419e]
 5: (KernelDevice::_aio_thread()+0xf9e) [0x560cadc1681e]
 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x560cadc24051]
 7: /lib64/libpthread.so.0(+0x81cf) [0x7f27915bb1cf]
 8: clone()
2022-04-09T06:52:05.042 INFO:scrape:1 jobs: ['6783080']
2022-04-09T06:52:05.043 INFO:scrape:suites: ['cluster/2-nodes/{base/{install', 'extra/{rbd-mirror}', 'mirror', 'msgr-failures/few', 'objectstore/bluestore-hybrid', 'policy/simple', 'rbd/mirror-thrash/{base/{openstack}', 'roles}', 'supported-random-distro$/{centos_8}}', 'workloads/rbd-mirror-snapshot-stress-workunit-minimum}']
2022-04-09T06:52:05.044 INFO:scrape:
2022-04-09T06:52:05.044 INFO:scrape:Dead: None
2022-04-09T06:52:05.045 INFO:scrape:1 jobs: ['6783173']
2022-04-09T06:52:05.046 INFO:scrape:
2022-04-09T06:52:05.047 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds
2022-04-09T06:52:05.048 INFO:scrape:1 jobs: ['6783204']
2022-04-09T06:52:05.048 INFO:scrape:suites: ['cluster/2-nodes/{base/{install', 'extra/{rbd-mirror}', 'mirror', 'msgr-failures/few', 'objectstore/bluestore-stupid', 'policy/none', 'rbd/mirror-thrash/{base/{openstack}', 'roles}', 'supported-random-distro$/{ubuntu_latest}}', 'workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}']
2022-04-09T06:52:05.049 INFO:scrape:
2022-04-09T06:52:05.049 INFO:scrape:Failure: Command failed (workunit test rbd/rbd_mirror_stress.sh) on smithi178 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=69ad68d989715754945f23c59c7f3abe9a8b52dc TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh'
2022-04-09T06:52:05.049 INFO:scrape:1 jobs: ['6783147']
2022-04-09T06:52:05.050 INFO:scrape:suites: ['cluster/2-nodes/{base/{install', 'extra/{rbd-mirror}', 'mirror', 'msgr-failures/few', 'objectstore/filestore-xfs', 'policy/simple', 'rbd/mirror-thrash/{base/{openstack}', 'roles}', 'supported-random-distro$/{ubuntu_latest}}', 'workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}']
2022-04-09T06:52:05.050 INFO:scrape:
2022-04-09T06:52:05.061 INFO:teuthology.results:Sending results to arthur.outhenin-chalandre@cern.ch: Test Run: mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/
failed:  22
dead:    1
running: 0
waiting: 0
queued:  0
passed:  167


Fail
=================================================================
[6783019]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-bitmap policy/none supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:37:13
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783019/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783019/

    "2022-04-08T23:45:55.888458+0000 mon.a (mon.0) 218 : cluster [WRN] Health
    check failed: Degraded data redundancy: 6/2862 objects degraded (0.210%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6783020]  rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:32:54
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783020/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783020/
sentry: https://sentry.ceph.com/organizations/ceph/?query=374864986d3d45d3af1d31ca32b4c288

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi167 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=69ad68d989715754945f23c59c7f3abe9a8b52dc
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1
    RBD_MIRROR_USE_RBD_MIRROR=1 USE_CLUSTER3=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh'

[6783038]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-hybrid policy/none supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:35:46
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783038/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783038/

    "2022-04-09T00:06:43.022408+0000 mon.a (mon.0) 206 : cluster [WRN] Health
    check failed: Degraded data redundancy: 1/2154 objects degraded (0.046%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6783060]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:40:32
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783060/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783060/

    "2022-04-09T00:32:12.154287+0000 mon.a (mon.0) 203 : cluster [WRN] Health
    check failed: Degraded data redundancy: 1/1856 objects degraded (0.054%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6783080]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/bluestore-hybrid policy/simple supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:52:55
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783080/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783080/

    "2022-04-09T01:19:03.689900+0000 mon.a (mon.0) 1458 : cluster [WRN] Health
    check failed: 1 osds down (OSD_DOWN)" in cluster log

[6783082]  rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-stupid policy/simple supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:52:12
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783082/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783082/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cc0d59bc75ce48f49202ef6a3c103105

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi163 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=69ad68d989715754945f23c59c7f3abe9a8b52dc
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1
    RBD_MIRROR_USE_RBD_MIRROR=1 USE_CLUSTER3=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh'

[6783084]  rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-bootstrap-workunit} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:10:54
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783084/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783084/

    Found coredumps on ubuntu@smithi036.front.sepia.ceph.com

[6783088]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/bluestore-comp-zlib policy/none supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:18:52
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783088/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783088/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a7ee4d17e13b401185d8fffac37be0f1

    "2022-04-09T01:00:03.948010+0000 mon.a (mon.0) 96 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6783090]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:39:45
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783090/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783090/

    "2022-04-09T01:02:48.461115+0000 mon.a (mon.0) 209 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/2450 objects degraded (0.082%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6783101]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-comp-snappy policy/none supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:41:07
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783101/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783101/

    "2022-04-09T04:23:36.409559+0000 mon.a (mon.0) 205 : cluster [WRN] Health
    check failed: Degraded data redundancy: 6/2044 objects degraded (0.294%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6783106]  rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-bootstrap-workunit} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:18:19
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783106/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783106/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1328ef06d3ab43f79dfd7e02f87bdb88

    "2022-04-09T04:27:06.643700+0000 mon.a (mon.0) 93 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6783110]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-bitmap policy/simple supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:40:18
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783110/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783110/

    "2022-04-09T04:30:13.555857+0000 mon.a (mon.0) 239 : cluster [WRN] Health
    check failed: Degraded data redundancy: 3/2598 objects degraded (0.115%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6783112]  rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:28:20
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783112/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783112/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b531b408dd8d47a5aec308ae68d1ee8a

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi178 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=69ad68d989715754945f23c59c7f3abe9a8b52dc
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1
    RBD_MIRROR_USE_RBD_MIRROR=1 USE_CLUSTER3=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh'

[6783126]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-comp-snappy policy/none supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:46:00
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783126/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783126/

    "2022-04-09T04:40:11.377961+0000 mon.a (mon.0) 203 : cluster [WRN] Health
    check failed: Degraded data redundancy: 1/334 objects degraded (0.299%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6783147]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/filestore-xfs policy/simple supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:43:15
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783147/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783147/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8239f7bdfa6046c9b8d3a47b23a430a5

    Command failed (workunit test rbd/rbd_mirror_stress.sh) on smithi178 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=69ad68d989715754945f23c59c7f3abe9a8b52dc
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image
    RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1
    RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh'

[6783151]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:38:46
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783151/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783151/

    "2022-04-09T05:10:17.638690+0000 mon.a (mon.0) 211 : cluster [WRN] Health
    check failed: Degraded data redundancy: 9/5898 objects degraded (0.153%), 2
    pgs degraded (PG_DEGRADED)" in cluster log

[6783153]  rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-snappy policy/none supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:45:50
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783153/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783153/

    Found coredumps on ubuntu@smithi081.front.sepia.ceph.com

[6783159]  rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-lz4 validator/memcheck workloads/rbd_mirror}
-----------------------------------------------------------------
time:   00:54:38
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783159/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783159/

    "2022-04-09T05:47:23.758564+0000 mon.a (mon.0) 1759 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)"
    in cluster log

[6783163]  rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-bitmap policy/simple supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:54:55
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783163/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783163/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a098406ca48543a5802b5186648644e5

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi189 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=69ad68d989715754945f23c59c7f3abe9a8b52dc
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1
    RBD_MIRROR_USE_RBD_MIRROR=1 USE_CLUSTER3=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh'

[6783184]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-comp-zstd policy/none supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:35:02
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783184/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783184/

    "2022-04-09T05:39:59.888611+0000 mon.a (mon.0) 210 : cluster [WRN] Health
    check failed: Degraded data redundancy: 8/3138 objects degraded (0.255%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6783196]  rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-zlib policy/none supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:30:22
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783196/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783196/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2ee065f101b14da4b71594b399473cee

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi170 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=69ad68d989715754945f23c59c7f3abe9a8b52dc
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1
    RBD_MIRROR_USE_RBD_MIRROR=1 USE_CLUSTER3=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh'

[6783204]  rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/bluestore-stupid policy/none supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:16:25
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783204/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783204/
sentry: https://sentry.ceph.com/organizations/ceph/?query=fea5b6c4caae40c0b075fc40ea14423d

    reached maximum tries (90) after waiting for 540 seconds



Dead
=================================================================
[6783173]  rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-stupid policy/none supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783173/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783173/

    Error reimaging machines: 'NoneType' object has no attribute '_fields'



Pass
=================================================================
[6783018] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-ha-workunit} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest}}
time:   00:13:36
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783018/

[6783021] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest}}
time:   00:25:12
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783021/

[6783022] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-stress-workunit} msgr-failures/few objectstore/bluestore-comp-snappy policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:56:52
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783022/

[6783023] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-journal-bootstrap-workunit} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest}}
time:   00:08:36
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783023/

[6783024] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:36:37
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783024/

[6783025] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-bootstrap-workunit} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8}}
time:   00:10:31
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783025/

[6783026] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-comp-zstd policy/none supported-random-distro$/{centos_8}}
time:   00:46:26
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783026/

[6783027] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/bluestore-hybrid policy/simple supported-random-distro$/{centos_8}}
time:   00:48:33
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783027/

[6783028] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest}}
time:   00:20:52
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783028/

[6783029] rbd/singleton/{all/rbd_mirror objectstore/bluestore-comp-snappy openstack supported-random-distro$/{ubuntu_latest}}
time:   00:43:23
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783029/

[6783030] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8}}
time:   00:51:44
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783030/

[6783031] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-stress-workunit} msgr-failures/few objectstore/bluestore-bitmap policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:58:23
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783031/

[6783032] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-config-key} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8}}
time:   00:29:50
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783032/

[6783033] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none supported-random-distro$/{centos_8}}
time:   00:37:21
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783033/

[6783034] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/bluestore-comp-zlib policy/none supported-random-distro$/{centos_8}}
time:   00:48:23
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783034/

[6783035] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-stupid validator/memcheck workloads/rbd_mirror}
time:   00:55:56
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783035/

[6783036] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-policy-simple} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest}}
time:   00:29:03
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783036/

[6783037] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple supported-random-distro$/{centos_8}}
time:   00:48:36
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783037/

[6783039] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple supported-random-distro$/{centos_8}}
time:   00:49:00
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783039/

[6783040] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8}}
time:   00:26:28
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783040/

[6783041] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-journal-bootstrap-workunit} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest}}
time:   00:08:25
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783041/

[6783042] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/filestore-xfs policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:39:01
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783042/

[6783043] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-bitmap policy/none supported-random-distro$/{centos_8}}
time:   00:47:18
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783043/

[6783044] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-exclusive-lock} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest}}
time:   00:19:56
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783044/

[6783045] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple supported-random-distro$/{centos_8}}
time:   00:48:53
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783045/

[6783046] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8}}
time:   00:21:32
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783046/

[6783047] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/bluestore-comp-snappy policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:47:59
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783047/

[6783048] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-journaling} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8}}
time:   00:21:02
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783048/

[6783049] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-minimum} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest}}
time:   00:20:24
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783049/

[6783050] rbd/singleton/{all/rbd_mirror objectstore/bluestore-comp-zlib openstack supported-random-distro$/{ubuntu_latest}}
time:   00:43:25
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783050/

[6783051] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8}}
time:   00:50:25
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783051/

[6783052] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-stress-workunit} msgr-failures/few objectstore/bluestore-hybrid policy/simple supported-random-distro$/{centos_8}}
time:   00:56:40
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783052/

[6783053] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:34:57
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783053/

[6783054] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest}}
time:   00:27:43
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783054/

[6783055] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-stupid policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:46:04
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783055/

[6783056] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/filestore-xfs policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:45:06
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783056/

[6783057] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-low-osd-mem-target validator/memcheck workloads/rbd_mirror}
time:   00:55:41
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783057/

[6783058] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/bluestore-bitmap policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:47:51
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783058/

[6783059] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-ha-workunit} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8}}
time:   00:14:56
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783059/

[6783061] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple supported-random-distro$/{centos_8}}
time:   00:57:41
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783061/

[6783062] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest}}
time:   00:26:08
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783062/

[6783063] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-stress-workunit} msgr-failures/few objectstore/bluestore-comp-zlib policy/none supported-random-distro$/{centos_8}}
time:   00:56:00
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783063/

[6783064] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple supported-random-distro$/{centos_8}}
time:   00:39:33
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783064/

[6783065] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-bootstrap-workunit} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8}}
time:   00:10:25
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783065/

[6783066] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-hybrid policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:44:53
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783066/

[6783067] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8}}
time:   00:21:49
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783067/

[6783068] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:48:22
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783068/

[6783069] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8}}
time:   00:21:21
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783069/

[6783070] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/bluestore-stupid policy/none supported-random-distro$/{centos_8}}
time:   00:48:10
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783070/

[6783071] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/filestore-xfs policy/simple supported-random-distro$/{centos_8}}
time:   00:42:54
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783071/

[6783072] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-minimum} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest}}
time:   00:19:53
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783072/

[6783073] rbd/singleton/{all/rbd_mirror objectstore/bluestore-comp-zstd openstack supported-random-distro$/{ubuntu_latest}}
time:   00:43:38
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783073/

[6783074] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_8}}
time:   00:52:31
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783074/

[6783075] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-stress-workunit} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:58:48
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783075/

[6783076] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/bluestore-comp-snappy policy/none supported-random-distro$/{centos_8}}
time:   00:36:36
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783076/

[6783077] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8}}
time:   00:29:23
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783077/

[6783078] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-policy-none} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest}}
time:   00:28:06
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783078/

[6783079] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-hybrid validator/memcheck workloads/rbd_mirror}
time:   00:55:42
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783079/

[6783081] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none supported-random-distro$/{centos_8}}
time:   00:38:51
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783081/

[6783083] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-journal-bootstrap-workunit} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8}}
time:   00:10:07
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783083/

[6783085] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:45:01
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783085/

[6783086] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_8}}
time:   00:21:40
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783086/

[6783087] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:48:52
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783087/

[6783089] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-journaling} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8}}
time:   00:21:30
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783089/

[6783091] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-minimum} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest}}
time:   00:19:07
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783091/

[6783092] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-hybrid policy/none supported-random-distro$/{centos_8}}
time:   00:48:00
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783092/

[6783093] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8}}
time:   00:51:38
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783093/

[6783094] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-stress-workunit} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple supported-random-distro$/{centos_8}}
time:   00:58:47
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783094/

[6783095] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-config-key} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8}}
time:   00:30:01
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783095/

[6783096] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/bluestore-stupid policy/none supported-random-distro$/{centos_8}}
time:   00:37:43
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783096/

[6783097] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest}}
time:   00:27:36
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783097/

[6783098] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-policy-none} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8}}
time:   00:30:04
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783098/

[6783099] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-zstd validator/memcheck workloads/rbd_mirror}
time:   00:55:04
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783099/

[6783100] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-policy-simple} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest}}
time:   00:27:29
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783100/

[6783102] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple supported-random-distro$/{centos_8}}
time:   00:50:07
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783102/

[6783103] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest}}
time:   00:24:44
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783103/

[6783104] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-stress-workunit} msgr-failures/few objectstore/bluestore-comp-zstd policy/none supported-random-distro$/{centos_8}}
time:   00:53:36
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783104/

[6783105] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-journal-bootstrap-workunit} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8}}
time:   00:10:59
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783105/

[6783107] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none supported-random-distro$/{centos_8}}
time:   00:46:40
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783107/

[6783108] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/bluestore-stupid policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:48:06
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783108/

[6783109] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-journaling} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest}}
time:   00:19:24
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783109/

[6783111] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-minimum} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8}}
time:   00:21:06
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783111/

[6783113] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8}}
time:   00:55:11
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783113/

[6783114] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-config-key} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest}}
time:   00:28:12
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783114/

[6783115] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-policy-none} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8}}
time:   00:29:02
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783115/

[6783116] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/bluestore-hybrid policy/none supported-random-distro$/{centos_8}}
time:   00:48:40
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783116/

[6783117] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-zlib validator/memcheck workloads/rbd_mirror}
time:   00:55:05
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783117/

[6783118] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-policy-simple} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8}}
time:   00:28:42
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783118/

[6783119] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-ha-workunit} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest}}
time:   00:13:54
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783119/

[6783120] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-stupid policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:36:25
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783120/

[6783121] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/filestore-xfs policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:49:27
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783121/

[6783122] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest}}
time:   00:24:47
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783122/

[6783123] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-journal-bootstrap-workunit} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8}}
time:   00:10:28
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783123/

[6783124] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple supported-random-distro$/{centos_8}}
time:   00:41:03
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783124/

[6783125] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-bootstrap-workunit} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8}}
time:   00:10:26
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783125/

[6783127] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8}}
time:   00:21:02
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783127/

[6783128] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_8}}
time:   00:21:45
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783128/

[6783129] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/bluestore-comp-zstd policy/none supported-random-distro$/{centos_8}}
time:   00:47:05
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783129/

[6783130] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-hybrid policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:37:39
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783130/

[6783131] rbd/singleton/{all/rbd_mirror objectstore/bluestore-stupid openstack supported-random-distro$/{centos_8}}
time:   00:44:38
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783131/

[6783132] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest}}
time:   00:54:11
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783132/

[6783133] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/filestore-xfs policy/none supported-random-distro$/{centos_8}}
time:   00:37:07
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783133/

[6783134] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8}}
time:   00:29:13
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783134/

[6783135] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-bitmap policy/simple supported-random-distro$/{centos_8}}
time:   00:50:18
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783135/

[6783136] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-policy-none} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8}}
time:   00:28:49
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783136/

[6783137] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:46:15
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783137/

[6783138] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-snappy validator/memcheck workloads/rbd_mirror}
time:   00:54:58
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783138/

[6783139] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-ha-workunit} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8}}
time:   00:15:22
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783139/

[6783140] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-comp-zlib policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:45:52
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783140/

[6783141] rbd/mirror-thrash/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit-multipeer} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple supported-random-distro$/{centos_8}}
time:   00:46:54
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783141/

[6783142] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_8}}
time:   00:25:37
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783142/

[6783143] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-journal-bootstrap-workunit} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8}}
time:   00:10:13
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783143/

[6783144] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple supported-random-distro$/{centos_8}}
time:   00:38:31
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783144/

[6783145] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-stupid policy/none supported-random-distro$/{centos_8}}
time:   00:47:53
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783145/

[6783146] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest}}
time:   00:20:03
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783146/

[6783148] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8}}
time:   00:22:08
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783148/

[6783149] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/bluestore-bitmap policy/none supported-random-distro$/{centos_8}}
time:   00:46:38
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783149/

[6783150] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-journaling} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest}}
time:   00:19:21
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783150/

[6783152] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-minimum} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest}}
time:   00:20:29
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783152/

[6783154] rbd/singleton/{all/rbd_mirror objectstore/filestore-xfs openstack supported-random-distro$/{centos_8}}
time:   00:44:38
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783154/

[6783155] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest}}
time:   00:52:51
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783155/

[6783156] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest}}
time:   00:27:15
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783156/

[6783157] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-hybrid policy/simple supported-random-distro$/{centos_8}}
time:   00:48:48
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783157/

[6783158] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none supported-random-distro$/{centos_8}}
time:   00:47:11
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783158/

[6783160] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-policy-simple} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8}}
time:   00:29:23
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783160/

[6783161] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/bluestore-stupid policy/simple supported-random-distro$/{centos_8}}
time:   00:50:00
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783161/

[6783162] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-ha-workunit} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest}}
time:   00:13:30
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783162/

[6783164] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest}}
time:   00:25:00
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783164/

[6783165] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-stress-workunit} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none supported-random-distro$/{centos_8}}
time:   00:54:23
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783165/

[6783166] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-comp-zlib policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:46:06
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783166/

[6783167] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8}}
time:   00:21:05
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783167/

[6783168] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:47:45
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783168/

[6783169] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest}}
time:   00:20:09
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783169/

[6783170] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/bluestore-hybrid policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:45:52
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783170/

[6783171] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-journaling} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest}}
time:   00:20:02
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783171/

[6783172] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-fsx-workunit} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple supported-random-distro$/{centos_8}}
time:   00:43:42
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783172/

[6783174] rbd/singleton/{all/rbd_mirror objectstore/bluestore-bitmap openstack supported-random-distro$/{centos_8}}
time:   00:45:10
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783174/

[6783175] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8}}
time:   00:49:56
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783175/

[6783176] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-config-key} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8}}
time:   00:29:28
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783176/

[6783177] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/bluestore-bitmap policy/none supported-random-distro$/{centos_8}}
time:   00:38:37
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783177/

[6783178] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest}}
time:   00:28:04
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783178/

[6783179] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-policy-none} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8}}
time:   00:29:42
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783179/

[6783180] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-bitmap validator/memcheck workloads/rbd_mirror}
time:   00:54:23
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783180/

[6783181] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-policy-simple} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8}}
time:   00:29:29
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783181/

[6783182] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple supported-random-distro$/{centos_8}}
time:   00:47:10
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783182/

[6783183] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-ha-workunit} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8}}
time:   00:15:35
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783183/

[6783185] rbd/mirror/{base/{openstack} cluster/3-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest}}
time:   00:25:50
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783185/

[6783186] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-stress-workunit} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:55:55
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783186/

[6783187] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-journal-bootstrap-workunit} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_8}}
time:   00:10:23
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783187/

[6783188] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/bluestore-stupid policy/simple supported-random-distro$/{centos_8}}
time:   00:39:24
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783188/

[6783189] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-bootstrap-workunit} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest}}
time:   00:08:12
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783189/

[6783190] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} msgr-failures/few objectstore/filestore-xfs policy/none supported-random-distro$/{centos_8}}
time:   00:49:48
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783190/

[6783191] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-exclusive-lock} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8}}
time:   00:21:02
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783191/

[6783192] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} msgr-failures/few objectstore/bluestore-bitmap policy/simple supported-random-distro$/{centos_8}}
time:   00:48:21
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783192/

[6783193] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:47:24
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783193/

[6783194] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-journaling} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8}}
time:   00:20:59
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783194/

[6783195] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-snapshot-workunit-minimum} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8}}
time:   00:20:53
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783195/

[6783197] rbd/singleton/{all/rbd_mirror objectstore/bluestore-comp-lz4 openstack supported-random-distro$/{centos_8}}
time:   00:44:30
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783197/

[6783198] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest}}
time:   00:52:00
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783198/

[6783199] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-stress-workunit} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple supported-random-distro$/{ubuntu_latest}}
time:   00:57:06
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783199/

[6783200] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-config-key} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8}}
time:   00:29:38
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783200/

[6783201] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-journal-workunit} msgr-failures/few objectstore/bluestore-hybrid policy/none supported-random-distro$/{ubuntu_latest}}
time:   00:34:10
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783201/

[6783202] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-min-compat-client-octopus} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest}}
time:   00:27:51
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783202/

[6783203] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-policy-none} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest}}
time:   00:28:06
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783203/

[6783205] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/filestore-xfs validator/memcheck workloads/rbd_mirror}
time:   00:55:05
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783205/

[6783206] rbd/mirror/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{mirror-extra} workloads/rbd-mirror-workunit-policy-simple} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8}}
time:   00:28:22
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783206/

[6783207] rbd/mirror-thrash/{base/{openstack} cluster/2-nodes/{base/{install mirror roles} extra/{rbd-mirror} workloads/rbd-mirror-snapshot-stress-workunit-minimum} msgr-failures/few objectstore/filestore-xfs policy/simple supported-random-distro$/{centos_8}}
time:   00:51:48
info:   http://pulpito.ceph.com/mrfreezeex-2022-04-08_23:03:08-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6783207/