2022-02-14T17:55:58.461 INFO:root:teuthology version: 1.1.0-7cebb3f2
2022-02-14T17:55:58.462 INFO:teuthology.results:seed: '3761'
2022-02-14T23:15:47.818 INFO:root:teuthology version: 1.1.0-7cebb3f2
2022-02-14T23:15:47.875 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-02-15T00:17:53.074 INFO:teuthology.results:Tests finished! gathering results...
2022-02-15T00:18:39.993 INFO:scrape:Found 169 jobs
2022-02-15T00:19:08.955 INFO:scrape:Found 8 distinct failure reasons
2022-02-15T00:19:08.956 INFO:scrape:Failure: 
2022-02-15T00:19:08.956 INFO:scrape:16 jobs: ['6682282', '6682317', '6682299', '6682194', '6682326', '6682206', '6682346', '6682270', '6682273', '6682329', '6682252', '6682223', '6682293', '6682198', '6682216', '6682232']
2022-02-15T00:19:08.957 INFO:scrape:suites intersection: ['cluster/{2-node', 'msgr-failures/few', 'openstack}']
2022-02-15T00:19:08.957 INFO:scrape:suites union: ['clients/mirror', 'clients/{mirror-extra', 'cluster/{2-node', 'mirror}', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'objectstore/filestore-xfs', 'openstack}', 'policy/none', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'rbd/mirror/{base/install', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-journal-workunit-multipeer}', 'workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer}']
2022-02-15T00:19:08.958 INFO:scrape:
2022-02-15T00:19:08.958 INFO:scrape:Failure: "2022-02-14T20:14:09.859641+0000 mon.a (mon.0) 203 : cluster [WRN] Health check failed: Degraded data redundancy: 2/3412 objects degraded (0.059%), 1 pg degraded (PG_DEGRADED)" in cluster log
2022-02-15T00:19:08.959 INFO:scrape:9 jobs: ['6682308', '6682349', '6682192', '6682257', '6682214', '6682344', '6682267', '6682222', '6682306']
2022-02-15T00:19:08.959 INFO:scrape:suites intersection: ['cluster/{2-node', 'msgr-failures/few', 'openstack}']
2022-02-15T00:19:08.960 INFO:scrape:suites union: ['clients/mirror', 'clients/{mirror-extra', 'cluster/{2-node', 'mirror}', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-stupid', 'openstack}', 'policy/none', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'rbd/mirror/{base/install', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-fsx-workunit}', 'workloads/rbd-mirror-ha-workunit}', 'workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}', 'workloads/rbd-mirror-snapshot-stress-workunit-minimum}', 'workloads/rbd-mirror-stress-workunit-min-compat-client-octopus}']
2022-02-15T00:19:08.960 INFO:scrape:
2022-02-15T00:19:08.961 INFO:scrape:Failure: reached maximum tries (90) after waiting for 540 seconds
2022-02-15T00:19:08.961 INFO:scrape:1 jobs: ['6682240']
2022-02-15T00:19:08.962 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-comp-lz4', 'openstack}', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-journal-workunit-multipeer}']
2022-02-15T00:19:08.963 INFO:scrape:
2022-02-15T00:19:08.963 INFO:scrape:Failure: Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi089 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c427b7b3d70b0814894a0b85f43e58f8da3b5e2a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh'
2022-02-15T00:19:08.964 INFO:scrape:2 jobs: ['6682359', '6682291']
2022-02-15T00:19:08.964 INFO:scrape:suites intersection: ['cluster/{2-node', 'msgr-failures/few', 'openstack}']
2022-02-15T00:19:08.965 INFO:scrape:suites union: ['clients/mirror', 'clients/{mirror-extra', 'cluster/{2-node', 'mirror}', 'msgr-failures/few', 'objectstore/bluestore-comp-lz4', 'objectstore/filestore-xfs', 'openstack}', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'rbd/mirror/{base/install', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-journal-workunit}', 'workloads/rbd-mirror-workunit-policy-simple}']
2022-02-15T00:19:08.965 INFO:scrape:
2022-02-15T00:19:08.966 INFO:scrape:Failure: Command failed (workunit test rbd/rbd_mirror_snapshot.sh) on smithi100 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c427b7b3d70b0814894a0b85f43e58f8da3b5e2a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' RBD_IMAGE_FEATURES=layering,exclusive-lock RBD_MIRROR_CONFIG_KEY=1 RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_snapshot.sh'
2022-02-15T00:19:08.967 INFO:scrape:2 jobs: ['6682292', '6682203']
2022-02-15T00:19:08.967 INFO:scrape:suites intersection: ['clients/{mirror-extra', 'cluster/{2-node', 'mirror}', 'msgr-failures/few', 'openstack}', 'rbd/mirror/{base/install']
2022-02-15T00:19:08.968 INFO:scrape:suites union: ['clients/{mirror-extra', 'cluster/{2-node', 'mirror}', 'msgr-failures/few', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-stupid', 'openstack}', 'rbd/mirror/{base/install', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-workunit-exclusive-lock}', 'workloads/rbd-mirror-snapshot-workunit-minimum}']
2022-02-15T00:19:08.968 INFO:scrape:
2022-02-15T00:19:08.968 INFO:scrape:Failure: Command failed (workunit test rbd/rbd_mirror_stress.sh) on smithi195 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c427b7b3d70b0814894a0b85f43e58f8da3b5e2a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_IMAGE_FEATURES=layering RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh'
2022-02-15T00:19:08.969 INFO:scrape:2 jobs: ['6682229', '6682266']
2022-02-15T00:19:08.969 INFO:scrape:suites intersection: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'openstack}', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'workloads/rbd-mirror-snapshot-stress-workunit-minimum}']
2022-02-15T00:19:08.970 INFO:scrape:suites union: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'openstack}', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-stress-workunit-minimum}']
2022-02-15T00:19:08.970 INFO:scrape:
2022-02-15T00:19:08.971 INFO:scrape:Failure: SSH connection to smithi138 was lost: 'sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.cluster1.client.mirror /home/ubuntu/cephtest/clone.cluster1.client.mirror'
2022-02-15T00:19:08.971 INFO:scrape:1 jobs: ['6682321']
2022-02-15T00:19:08.972 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-hybrid', 'openstack}', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{centos_8}', 'workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}']
2022-02-15T00:19:08.972 INFO:scrape:
2022-02-15T00:19:08.973 INFO:scrape:Failure: Command failed (workunit test rbd/test_rbd_mirror.sh) on smithi103 with status 139: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c427b7b3d70b0814894a0b85f43e58f8da3b5e2a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 VALGRIND=\'--tool=memcheck --leak-check=full\' adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_rbd_mirror.sh'
2022-02-15T00:19:08.973 INFO:scrape:1 jobs: ['6682305']
2022-02-15T00:19:08.974 INFO:scrape:suites: ['centos_latest', 'clusters/{fixed-1', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rbd/valgrind/{base/install', 'validator/memcheck', 'workloads/rbd_mirror}']
2022-02-15T00:19:08.974 INFO:scrape:
2022-02-15T00:19:08.982 INFO:teuthology.results:Sending results to arthur.outhenin-chalandre@cern.ch: Test Run: mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/
failed:  33
dead:    1
running: 0
waiting: 0
queued:  0
passed:  135


Fail
=================================================================
[6682192]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit}
-----------------------------------------------------------------
time:   00:40:44
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682192/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682192/

    "2022-02-14T18:13:45.190071+0000 mon.a (mon.0) 201 : cluster [WRN] Health
    check failed: Degraded data redundancy: 3/2646 objects degraded (0.113%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6682194]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit-multipeer}
-----------------------------------------------------------------
time:   00:07:05
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682194/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682194/
sentry: https://sentry.ceph.com/organizations/ceph/?query=94768f77911c45a4a00b4f80ed2bda32

[6682198]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer}
-----------------------------------------------------------------
time:   00:09:17
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682198/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682198/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5cedb61cc4c24b3591f567084c1484f7

[6682203]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-minimum}
-----------------------------------------------------------------
time:   00:13:56
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682203/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682203/
sentry: https://sentry.ceph.com/organizations/ceph/?query=83b869e29b3741dd82b1e025df055763

    Command failed (workunit test rbd/rbd_mirror_snapshot.sh) on smithi154 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c427b7b3d70b0814894a0b85f43e58f8da3b5e2a
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    RBD_IMAGE_FEATURES=layering RBD_MIRROR_CONFIG_KEY=1 RBD_MIRROR_INSTANCES=4
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_snapshot.sh'

[6682206]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-workunit-multipeer}
-----------------------------------------------------------------
time:   00:09:09
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682206/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682206/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0675b0a66bae409a8f00452180389d8e

[6682214]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit}
-----------------------------------------------------------------
time:   00:47:47
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682214/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682214/

    "2022-02-14T18:41:57.335851+0000 mon.a (mon.0) 205 : cluster [WRN] Health
    check failed: Degraded data redundancy: 4/2362 objects degraded (0.169%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6682216]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer}
-----------------------------------------------------------------
time:   00:08:08
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682216/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682216/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0caa871b6cc1414298860b8c9621adf3

[6682222]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus}
-----------------------------------------------------------------
time:   00:16:20
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682222/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682222/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4a8396c757fb4df5889ac80d21c585f9

    "2022-02-14T18:47:01.122879+0000 mon.a (mon.0) 90 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6682223]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit-multipeer}
-----------------------------------------------------------------
time:   00:07:16
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682223/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682223/
sentry: https://sentry.ceph.com/organizations/ceph/?query=51d9a991db9c4b1dbba56b29be49b3ad

[6682229]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
-----------------------------------------------------------------
time:   00:17:14
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682229/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682229/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ccda822f93ca42868fb7f738b912ebfb

    Command failed (workunit test rbd/rbd_mirror_stress.sh) on smithi195 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c427b7b3d70b0814894a0b85f43e58f8da3b5e2a
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image
    RBD_IMAGE_FEATURES=layering RBD_MIRROR_INSTANCES=4
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-
    ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /ho
    me/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror
    _stress.sh'

[6682232]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-workunit-multipeer}
-----------------------------------------------------------------
time:   00:08:05
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682232/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682232/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2aae9a4c697a4b689c9fde846f635d75

[6682240]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit-multipeer}
-----------------------------------------------------------------
time:   00:16:10
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682240/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682240/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d9996acce8e24ca999aee8e850451aa8

    reached maximum tries (90) after waiting for 540 seconds

[6682252]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer}
-----------------------------------------------------------------
time:   00:07:53
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682252/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682252/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0b9d33e43d9747bdb60f3dd9ae5014b7

[6682257]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit}
-----------------------------------------------------------------
time:   00:16:02
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682257/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682257/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ababd38c98a443bc88faf86a1974fbce

    "2022-02-14T19:16:57.576093+0000 mon.a (mon.0) 90 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6682266]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
-----------------------------------------------------------------
time:   00:50:30
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682266/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682266/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5a6f27fc2c71428fb7e29beabcbaabb6

    Command failed (workunit test rbd/rbd_mirror_stress.sh) on smithi187 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c427b7b3d70b0814894a0b85f43e58f8da3b5e2a
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image
    RBD_IMAGE_FEATURES=layering RBD_MIRROR_INSTANCES=4
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-
    ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /ho
    me/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror
    _stress.sh'

[6682267]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-ha-workunit}
-----------------------------------------------------------------
time:   00:16:05
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682267/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682267/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f88293c3c8644b95a28f96dab2bc9e81

    "2022-02-14T19:31:28.566816+0000 mon.a (mon.0) 88 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6682270]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit-multipeer}
-----------------------------------------------------------------
time:   00:07:23
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682270/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682270/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7b2912483969479e883c217151b5cf37

[6682273]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer}
-----------------------------------------------------------------
time:   00:07:11
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682273/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682273/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0aa75730f9d94f7f8f4c66a90b413de9

[6682282]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-workunit-multipeer}
-----------------------------------------------------------------
time:   00:08:12
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682282/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682282/
sentry: https://sentry.ceph.com/organizations/ceph/?query=cba2b7481f4849ada7f559226271faf4

[6682291]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit}
-----------------------------------------------------------------
time:   00:37:42
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682291/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682291/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7be1d3d25a534ce7a3516ff02a41677a

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi157 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c427b7b3d70b0814894a0b85f43e58f8da3b5e2a
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1
    RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh'

[6682292]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-exclusive-lock}
-----------------------------------------------------------------
time:   00:14:59
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682292/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682292/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3f12e6cabd1847269d3fd0ae0ab45599

    Command failed (workunit test rbd/rbd_mirror_snapshot.sh) on smithi100 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c427b7b3d70b0814894a0b85f43e58f8da3b5e2a
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    RBD_IMAGE_FEATURES=layering,exclusive-lock RBD_MIRROR_CONFIG_KEY=1
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_snapsho
    t.sh'

[6682293]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer}
-----------------------------------------------------------------
time:   00:08:02
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682293/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682293/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2483773232074e8b97fde0376303fd52

[6682299]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit-multipeer}
-----------------------------------------------------------------
time:   00:07:01
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682299/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682299/
sentry: https://sentry.ceph.com/organizations/ceph/?query=637ec010362a44ed89d315a1152c9925

[6682305]  rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-snappy validator/memcheck workloads/rbd_mirror}
-----------------------------------------------------------------
time:   00:26:48
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682305/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682305/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bd7c946ff7544540a334c5edf04298ff

    Command failed (workunit test rbd/test_rbd_mirror.sh) on smithi103 with
    status 139: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c427b7b3d70b0814894a0b85f43e58f8da3b5e2a
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 VALGRIND=\'--tool=memcheck --leak-
    check=full\' adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_rbd_mirror.sh'

[6682306]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
-----------------------------------------------------------------
time:   00:48:00
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682306/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682306/

    "2022-02-14T20:11:41.070215+0000 mon.a (mon.0) 206 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg inactive, 1 pg peering
    (PG_AVAILABILITY)" in cluster log

[6682308]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit}
-----------------------------------------------------------------
time:   00:39:22
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682308/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682308/

    "2022-02-14T20:14:09.859641+0000 mon.a (mon.0) 203 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/3412 objects degraded (0.059%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6682317]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-workunit-multipeer}
-----------------------------------------------------------------
time:   00:08:18
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682317/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682317/
sentry: https://sentry.ceph.com/organizations/ceph/?query=715e086f174d4777869b102c1f42f8d2

[6682326]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit-multipeer}
-----------------------------------------------------------------
time:   00:07:20
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682326/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682326/
sentry: https://sentry.ceph.com/organizations/ceph/?query=383a718514ee480088e848d479f12c85

[6682329]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-fast-diff-multipeer}
-----------------------------------------------------------------
time:   00:07:44
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682329/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682329/
sentry: https://sentry.ceph.com/organizations/ceph/?query=aeba43a57c7f4f8daddfdac4074a7625

[6682344]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit}
-----------------------------------------------------------------
time:   00:42:27
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682344/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682344/

    "2022-02-14T23:26:56.637884+0000 mon.a (mon.0) 206 : cluster [WRN] Health
    check failed: Degraded data redundancy: 6/2272 objects degraded (0.264%), 2
    pgs degraded (PG_DEGRADED)" in cluster log

[6682346]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit-multipeer}
-----------------------------------------------------------------
time:   00:07:29
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682346/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682346/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bed5264b4f914dd8be41d7e15e1d27b7

[6682349]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
-----------------------------------------------------------------
time:   00:16:29
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682349/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682349/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6174ec7249194f19b5372e2be1a1253a

    "2022-02-14T23:26:13.691932+0000 mon.a (mon.0) 90 : cluster [WRN] Health
    check failed: Degraded data redundancy: 2/4 objects degraded (50.000%), 1
    pg degraded (PG_DEGRADED)" in cluster log

[6682359]  rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-policy-simple}
-----------------------------------------------------------------
time:   00:15:12
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682359/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682359/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b47ee8dab90848809b4611b44106583c

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi089 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c427b7b3d70b0814894a0b85f43e58f8da3b5e2a
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal
    .sh'



Dead
=================================================================
[6682321]  rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
-----------------------------------------------------------------
time:   00:59:17
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682321/
log:    http://qa-proxy.ceph.com/teuthology/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682321/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bcaf546e65954e7490196a7e5ce9ef43

    SSH connection to smithi138 was lost: 'sudo rm -rf --
    /home/ubuntu/cephtest/workunits.list.cluster1.client.mirror
    /home/ubuntu/cephtest/clone.cluster1.client.mirror'



Pass
=================================================================
[6682193] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-stress-workunit}
time:   00:59:22
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682193/

[6682195] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-bootstrap-workunit}
time:   00:09:03
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682195/

[6682196] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-exclusive-lock}
time:   00:21:14
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682196/

[6682197] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:46:49
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682197/

[6682199] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:48:30
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682199/

[6682200] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-fast-diff}
time:   00:20:49
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682200/

[6682201] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:49:40
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682201/

[6682202] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-journaling}
time:   00:21:28
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682202/

[6682204] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-stress-workunit}
time:   01:01:17
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682204/

[6682205] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus}
time:   00:51:35
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682205/

[6682207] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-config-key}
time:   00:29:00
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682207/

[6682208] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit}
time:   00:38:19
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682208/

[6682209] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-policy-none}
time:   00:27:47
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682209/

[6682210] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:49:55
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682210/

[6682211] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-stupid validator/memcheck workloads/rbd_mirror}
time:   00:54:59
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682211/

[6682212] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:50:22
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682212/

[6682213] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} workloads/rbd-mirror-ha-workunit}
time:   00:15:02
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682213/

[6682215] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-exclusive-lock}
time:   00:20:32
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682215/

[6682217] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:51:44
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682217/

[6682218] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:37
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682218/

[6682219] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-journaling}
time:   00:21:07
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682219/

[6682220] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit}
time:   00:35:24
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682220/

[6682221] rbd/singleton/{all/rbd_mirror objectstore/bluestore-comp-zlib openstack supported-random-distro$/{centos_8}}
time:   00:44:23
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682221/

[6682224] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-config-key}
time:   00:27:59
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682224/

[6682225] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:47:47
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682225/

[6682226] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-policy-none}
time:   00:29:05
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682226/

[6682227] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:49:03
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682227/

[6682228] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-low-osd-mem-target validator/memcheck workloads/rbd_mirror}
time:   00:54:21
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682228/

[6682230] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8} workloads/rbd-mirror-ha-workunit}
time:   00:14:28
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682230/

[6682231] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-bootstrap-workunit}
time:   00:09:34
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682231/

[6682233] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-bootstrap-workunit}
time:   00:09:48
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682233/

[6682234] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:45:49
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682234/

[6682235] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:26
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682235/

[6682236] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit}
time:   00:37:31
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682236/

[6682237] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-minimum}
time:   00:20:07
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682237/

[6682238] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-stress-workunit}
time:   00:54:06
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682238/

[6682239] rbd/singleton/{all/rbd_mirror objectstore/bluestore-comp-zstd openstack supported-random-distro$/{centos_8}}
time:   00:44:12
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682239/

[6682241] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-config-key}
time:   00:28:06
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682241/

[6682242] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit}
time:   00:37:41
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682242/

[6682243] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-min-compat-client-octopus}
time:   00:27:43
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682243/

[6682244] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-policy-none}
time:   00:30:20
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682244/

[6682245] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:50:29
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682245/

[6682246] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-hybrid validator/memcheck workloads/rbd_mirror}
time:   00:54:53
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682246/

[6682247] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-policy-simple}
time:   00:28:34
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682247/

[6682248] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-ha-workunit}
time:   00:13:39
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682248/

[6682249] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-fsx-workunit}
time:   00:38:18
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682249/

[6682250] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-stress-workunit}
time:   00:56:15
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682250/

[6682251] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-bootstrap-workunit}
time:   00:10:17
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682251/

[6682253] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:51:25
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682253/

[6682254] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-fast-diff}
time:   00:20:45
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682254/

[6682255] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:52
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682255/

[6682256] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-journaling}
time:   00:19:30
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682256/

[6682258] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-minimum}
time:   00:20:26
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682258/

[6682259] rbd/singleton/{all/rbd_mirror objectstore/bluestore-hybrid openstack supported-random-distro$/{ubuntu_latest}}
time:   00:42:59
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682259/

[6682260] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-config-key}
time:   00:27:58
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682260/

[6682261] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-workunit}
time:   00:36:56
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682261/

[6682262] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-min-compat-client-octopus}
time:   00:28:12
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682262/

[6682263] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:50:45
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682263/

[6682264] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:50:11
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682264/

[6682265] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-zstd validator/memcheck workloads/rbd_mirror}
time:   00:54:46
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682265/

[6682268] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-stress-workunit}
time:   00:54:51
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682268/

[6682269] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-bootstrap-workunit}
time:   00:09:17
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682269/

[6682271] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-exclusive-lock}
time:   00:19:02
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682271/

[6682272] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:46:24
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682272/

[6682274] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:50:22
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682274/

[6682275] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-fast-diff}
time:   00:19:39
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682275/

[6682276] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:49:16
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682276/

[6682277] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-journaling}
time:   00:22:04
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682277/

[6682278] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit}
time:   00:35:47
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682278/

[6682279] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-stress-workunit}
time:   00:55:45
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682279/

[6682280] rbd/singleton/{all/rbd_mirror objectstore/bluestore-low-osd-mem-target openstack supported-random-distro$/{ubuntu_latest}}
time:   00:43:28
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682280/

[6682281] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus}
time:   00:53:18
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682281/

[6682283] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-config-key}
time:   00:28:59
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682283/

[6682284] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-min-compat-client-octopus}
time:   00:27:06
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682284/

[6682285] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:46:45
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682285/

[6682286] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-zlib validator/memcheck workloads/rbd_mirror}
time:   00:54:04
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682286/

[6682287] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-policy-simple}
time:   00:27:39
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682287/

[6682288] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:13
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682288/

[6682289] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-ha-workunit}
time:   00:13:53
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682289/

[6682290] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-bootstrap-workunit}
time:   00:09:55
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682290/

[6682294] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:51:01
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682294/

[6682295] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-journaling}
time:   00:18:54
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682295/

[6682296] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-stress-workunit}
time:   00:57:07
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682296/

[6682297] rbd/singleton/{all/rbd_mirror objectstore/bluestore-stupid openstack supported-random-distro$/{centos_8}}
time:   00:44:20
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682297/

[6682298] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus}
time:   00:52:33
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682298/

[6682300] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-config-key}
time:   00:29:32
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682300/

[6682301] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-workunit}
time:   00:37:18
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682301/

[6682302] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:47:07
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682302/

[6682303] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-policy-none}
time:   00:28:43
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682303/

[6682304] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:49:06
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682304/

[6682307] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} workloads/rbd-mirror-ha-workunit}
time:   00:14:09
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682307/

[6682309] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-bootstrap-workunit}
time:   00:09:30
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682309/

[6682310] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit}
time:   00:38:13
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682310/

[6682311] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-exclusive-lock}
time:   00:20:25
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682311/

[6682312] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:46:32
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682312/

[6682313] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:52:16
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682313/

[6682314] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-fast-diff}
time:   00:21:04
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682314/

[6682315] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-journaling}
time:   00:21:37
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682315/

[6682316] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-fsx-workunit}
time:   00:43:57
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682316/

[6682318] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-config-key}
time:   00:27:58
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682318/

[6682319] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-workunit}
time:   00:38:12
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682319/

[6682320] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-min-compat-client-octopus}
time:   00:28:42
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682320/

[6682322] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-lz4 validator/memcheck workloads/rbd_mirror}
time:   00:56:35
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682322/

[6682323] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-policy-simple}
time:   00:28:38
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682323/

[6682324] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:04
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682324/

[6682325] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} workloads/rbd-mirror-ha-workunit}
time:   00:14:51
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682325/

[6682327] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-exclusive-lock}
time:   00:21:35
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682327/

[6682328] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:46:57
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682328/

[6682330] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:53:03
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682330/

[6682331] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-fast-diff}
time:   00:20:22
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682331/

[6682332] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:48:17
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682332/

[6682333] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-journaling}
time:   00:20:24
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682333/

[6682334] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit}
time:   00:40:45
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682334/

[6682335] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-minimum}
time:   00:20:00
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682335/

[6682336] rbd/singleton/{all/rbd_mirror objectstore/bluestore-bitmap openstack supported-random-distro$/{centos_8}}
time:   00:44:14
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682336/

[6682337] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus}
time:   00:53:12
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682337/

[6682338] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-config-key}
time:   00:28:25
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682338/

[6682339] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-min-compat-client-octopus}
time:   00:28:36
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682339/

[6682340] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-policy-none}
time:   00:28:35
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682340/

[6682341] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-bitmap validator/memcheck workloads/rbd_mirror}
time:   00:54:30
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682341/

[6682342] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-policy-simple}
time:   00:28:55
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682342/

[6682343] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:29
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682343/

[6682345] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-stress-workunit}
time:   00:58:38
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682345/

[6682347] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-exclusive-lock}
time:   00:19:15
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682347/

[6682348] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:47:41
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682348/

[6682350] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:39
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682350/

[6682351] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-journaling}
time:   00:20:13
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682351/

[6682352] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-minimum}
time:   00:19:17
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682352/

[6682353] rbd/singleton/{all/rbd_mirror objectstore/bluestore-comp-lz4 openstack supported-random-distro$/{ubuntu_latest}}
time:   00:43:26
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682353/

[6682354] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus}
time:   00:54:11
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682354/

[6682355] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-min-compat-client-octopus}
time:   00:28:35
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682355/

[6682356] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}
time:   00:47:22
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682356/

[6682357] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff}
time:   00:49:00
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682357/

[6682358] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/filestore-xfs validator/memcheck workloads/rbd_mirror}
time:   00:55:21
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682358/

[6682360] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}
time:   00:47:31
info:   http://pulpito.ceph.com/mrfreezeex-2022-02-14_17:14:05-rbd-wip-mrfreezeex-multipeer-distro-default-smithi/6682360/