2020-05-04T13:24:08.112 INFO:root:teuthology version: 1.0.0-58684fc5
2020-05-04T13:24:08.115 INFO:teuthology.results:seed: '1440'
2020-05-04T14:12:22.185 INFO:root:teuthology version: 1.0.0-58684fc5
2020-05-04T14:12:22.206 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-05-04T15:08:23.488 INFO:teuthology.results:Tests finished! gathering results...
2020-05-04T15:08:23.600 INFO:teuthology.results:Sending results to to.my.trociny@gmail.com: Test Run: trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/
failed:  43
dead:    0
running: 0
waiting: 0
queued:  0
passed:  17


Fail
=================================================================
[5021876]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-ha-workunit.yaml}
-----------------------------------------------------------------
time:   00:10:48
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021876/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021876/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=99f1af0d7bd945c3b1ab2c0c786f0cf5

    Command failed (workunit test rbd/rbd_mirror_ha.sh) on smithi092 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_ha.sh'

[5021877]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-journal-workunit.yaml}
-----------------------------------------------------------------
time:   00:08:16
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021877/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021877/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=a501aa5d9f3b42fd8a2e90e75860eec3

    No module named 'util'

[5021878]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-lz4.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-snapshot-workunit.yaml}
-----------------------------------------------------------------
time:   00:18:47
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021878/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021878/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=a310ff23bd5f4ecd96159777047b831f

    Command failed (workunit test rbd/rbd_mirror_snapshot.sh) on smithi201 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_CONFIG_KEY=1 RBD_MIRROR_INSTANCES=4
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_snapshot.sh'

[5021879]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-lz4.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-stress-workunit.yaml}
-----------------------------------------------------------------
time:   00:07:52
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021879/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021879/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=0a10382198a943e1a52f5ffabbc0bf88

    No module named 'util'

[5021881]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zlib.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-workunit-config-key.yaml}
-----------------------------------------------------------------
time:   00:11:12
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021881/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021881/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=23e3ac0659724e65ae61c55adf813596

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi003 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_CONFIG_KEY=1 RBD_MIRROR_INSTANCES=4
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh'

[5021882]  rbd/singleton/{all/rbd_mirror.yaml objectstore/bluestore-bitmap.yaml openstack.yaml supported-random-distro$/{rhel_8.yaml}}
-----------------------------------------------------------------
time:   00:34:34
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021882/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021882/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=6289936ffb8343a3a132409c1e6a85a5

    Command failed (workunit test rbd/test_rbd_mirror.sh) on smithi076 with
    status 134: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_rbd_mirror.sh'

[5021883]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zstd.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-workunit-min-compat-client-octopus.yaml}
-----------------------------------------------------------------
time:   00:10:58
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021883/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021883/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=4ed486479b2b48c4b4245cefe5b35447

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi122 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_MIN_COMPAT_CLIENT=octopus RBD_MIRROR_USE_EXISTING_CLUSTER=1
    adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout
    3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_
    mirror_journal.sh'

[5021884]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zlib.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-journal-workunit.yaml}
-----------------------------------------------------------------
time:   00:08:13
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021884/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021884/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=c857b4f8b9d7490dae072dfc52211575

    No module named 'util'

[5021885]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-workunit-policy-none.yaml}
-----------------------------------------------------------------
time:   00:14:16
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021885/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021885/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=8863f76a726c4bc6b04a154703e7b4b7

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi013 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal
    .sh'

[5021886]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zstd.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-stress-workunit.yaml}
-----------------------------------------------------------------
time:   00:08:29
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021886/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021886/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=f27595fdbc624ccd91f0d64bbf9c06c0

    No module named 'util'

[5021887]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-stupid.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-workunit-policy-simple.yaml}
-----------------------------------------------------------------
time:   00:14:08
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021887/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021887/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=b4ec8b5304544dae9cb69ff6cd41b067

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi197 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal
    .sh'

[5021888]  rbd/valgrind/{base/install.yaml centos_latest.yaml clusters/{fixed-1.yaml openstack.yaml} objectstore/bluestore-stupid.yaml validator/memcheck.yaml workloads/rbd_mirror.yaml}
-----------------------------------------------------------------
time:   00:59:14
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021888/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021888/

    "2020-05-04T14:52:31.742707+0000 mon.a (mon.0) 2361 : cluster [WRN] Health
    check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering
    (PG_AVAILABILITY)" in cluster log

[5021890]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-avl.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-ha-workunit.yaml}
-----------------------------------------------------------------
time:   00:14:06
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021890/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021890/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=e288b4a9df1d43f294defbc73aac7c01

    Command failed (workunit test rbd/rbd_mirror_ha.sh) on smithi049 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_ha.sh'

[5021891]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-stupid.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-journal-workunit.yaml}
-----------------------------------------------------------------
time:   00:10:55
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021891/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021891/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=8483dc872665456c955d3a6373ada791

    No module named 'util'

[5021893]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-stress-workunit.yaml}
-----------------------------------------------------------------
time:   00:10:55
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021893/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021893/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=65ff3bab6d3d49168d711e411df6c72d

    No module named 'util'

[5021895]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-snappy.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-workunit-config-key.yaml}
-----------------------------------------------------------------
time:   00:11:14
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021895/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021895/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=e666abf82fe24f85b71966ea54389bea

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi019 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_CONFIG_KEY=1 RBD_MIRROR_INSTANCES=4
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh'

[5021896]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-avl.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-stress-workunit.yaml}
-----------------------------------------------------------------
time:   00:10:41
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021896/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021896/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=214e42301c1544d0b5efcf4a5ccb2f1a

    No module named 'util'

[5021898]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zlib.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-workunit-min-compat-client-octopus.yaml}
-----------------------------------------------------------------
time:   00:14:09
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021898/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021898/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=5e6fc22277b34184b4bb9a5c3fc7888c

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi014 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_MIN_COMPAT_CLIENT=octopus RBD_MIRROR_USE_EXISTING_CLUSTER=1
    adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout
    3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_
    mirror_journal.sh'

[5021899]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zstd.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-workunit-policy-none.yaml}
-----------------------------------------------------------------
time:   00:13:55
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021899/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021899/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=a47289bd12254a93a709c834e4b22401

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi172 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal
    .sh'

[5021900]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-lz4.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-journal-workunit.yaml}
-----------------------------------------------------------------
time:   00:08:29
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021900/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021900/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=d3d7d141fec24650a4f0a89fa962e096

    No module named 'util'

[5021901]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-workunit-policy-simple.yaml}
-----------------------------------------------------------------
time:   00:10:49
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021901/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021901/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=f691aaf539b747d1905a80a8b6bde911

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi148 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal
    .sh'

[5021904]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-snappy.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-stress-workunit.yaml}
-----------------------------------------------------------------
time:   00:08:23
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021904/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021904/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=a0659fbe9b1543eca122ab9ffdf3bc92

    No module named 'util'

[5021905]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-ha-workunit.yaml}
-----------------------------------------------------------------
time:   00:11:08
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021905/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021905/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=08738539f4f14e16b7a09bd9a7d17d8e

    Command failed (workunit test rbd/rbd_mirror_ha.sh) on smithi096 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_ha.sh'

[5021907]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zstd.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-journal-workunit.yaml}
-----------------------------------------------------------------
time:   00:07:42
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021907/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021907/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=7c30824c6f80429e8ee2664cbcd274f0

    No module named 'util'

[5021909]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-lz4.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-workunit-config-key.yaml}
-----------------------------------------------------------------
time:   00:11:46
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021909/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021909/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=459ce5de2b2c4f73aa3f710cba1555e3

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi124 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_CONFIG_KEY=1 RBD_MIRROR_INSTANCES=4
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh'

[5021910]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-stress-workunit.yaml}
-----------------------------------------------------------------
time:   00:07:52
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021910/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021910/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=40c4e54ffc6542959dde28c3982cbaba

    No module named 'util'

[5021912]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-snappy.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-workunit-min-compat-client-octopus.yaml}
-----------------------------------------------------------------
time:   00:11:25
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021912/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021912/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=e720cac74ad9446fb992581b536c64fd

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi122 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_MIN_COMPAT_CLIENT=octopus RBD_MIRROR_USE_EXISTING_CLUSTER=1
    adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout
    3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_
    mirror_journal.sh'

[5021913]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zlib.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-workunit-policy-none.yaml}
-----------------------------------------------------------------
time:   00:11:39
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021913/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021913/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=84cb8071ca924afbb45c4fc4a6982fdf

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi017 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal
    .sh'

[5021914]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-journal-workunit.yaml}
-----------------------------------------------------------------
time:   00:07:57
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021914/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021914/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=f124d67ae503491994fc5e76cb8615da

    No module named 'util'

[5021915]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zstd.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-workunit-policy-simple.yaml}
-----------------------------------------------------------------
time:   00:14:19
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021915/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021915/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=83d8bfdd6abb4a56b071a50b827d90bf

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi069 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal
    .sh'

[5021918]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-avl.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-journal-workunit.yaml}
-----------------------------------------------------------------
time:   00:11:07
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021918/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021918/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=f97ef918fe0d4f93b857aaf1905bcde1

    No module named 'util'

[5021919]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-stupid.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-ha-workunit.yaml}
-----------------------------------------------------------------
time:   00:10:47
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021919/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021919/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=4cc838c120634806b1c30d05353c3803

    Command failed (workunit test rbd/rbd_mirror_ha.sh) on smithi104 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_ha.sh'

[5021920]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-stress-workunit.yaml}
-----------------------------------------------------------------
time:   00:08:10
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021920/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021920/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=173a14f5c97344c09639cefd1d863624

    No module named 'util'

[5021923]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-workunit-config-key.yaml}
-----------------------------------------------------------------
time:   00:11:00
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021923/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021923/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=f97fc95da41a4e49b90925e52f8d8ac8

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi058 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_CONFIG_KEY=1 RBD_MIRROR_INSTANCES=4
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal.sh'

[5021924]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-snappy.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-journal-workunit.yaml}
-----------------------------------------------------------------
time:   00:08:26
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021924/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021924/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=f7d466e326d3466b845c607f810d4909

    No module named 'util'

[5021926]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-lz4.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-workunit-min-compat-client-octopus.yaml}
-----------------------------------------------------------------
time:   00:11:33
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021926/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021926/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=dc907703c1ca4af98f4f5675d6bebb3f

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi167 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_MIN_COMPAT_CLIENT=octopus RBD_MIRROR_USE_EXISTING_CLUSTER=1
    adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout
    3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_
    mirror_journal.sh'

[5021927]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zlib.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-stress-workunit.yaml}
-----------------------------------------------------------------
time:   00:07:46
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021927/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021927/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=fe6d308f74fe4409991fd6722778809e

    No module named 'util'

[5021928]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-snappy.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-workunit-policy-none.yaml}
-----------------------------------------------------------------
time:   00:11:09
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021928/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021928/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=bd016a6ae1074e7f9d04c834e587d8c1

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi155 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal
    .sh'

[5021929]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zlib.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-workunit-policy-simple.yaml}
-----------------------------------------------------------------
time:   00:11:04
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021929/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021929/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=f6577c8e575b418bb52bd77cfdbf794b

    Command failed (workunit test rbd/rbd_mirror_journal.sh) on smithi063 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_journal
    .sh'

[5021930]  rbd/valgrind/{base/install.yaml centos_latest.yaml clusters/{fixed-1.yaml openstack.yaml} objectstore/bluestore-comp-zlib.yaml validator/memcheck.yaml workloads/rbd_mirror.yaml}
-----------------------------------------------------------------
time:   00:51:43
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021930/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021930/

    "2020-05-04T14:49:05.737553+0000 mon.a (mon.0) 1622 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg inactive, 1 pg peering
    (PG_AVAILABILITY)" in cluster log

[5021932]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml policy/none.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-journal-workunit.yaml}
-----------------------------------------------------------------
time:   00:11:10
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021932/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021932/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=697d789d6502450eb6ef749f46363f3b

    No module named 'util'

[5021933]  rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-ha-workunit.yaml}
-----------------------------------------------------------------
time:   00:11:49
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021933/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021933/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=2dc2242919ab41249c6beb2b3c7feae8

    Command failed (workunit test rbd/rbd_mirror_ha.sh) on smithi138 with
    status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=ab2488773626d40fb1f8345a99f4bbd6712c0fd4
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\'
    RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clo
    ne.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_ha.sh'

[5021934]  rbd/mirror-thrash/{base/install.yaml clients/mirror.yaml cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-stupid.yaml policy/simple.yaml rbd-mirror/four-per-cluster.yaml supported-random-distro$/{rhel_8.yaml} workloads/rbd-mirror-stress-workunit.yaml}
-----------------------------------------------------------------
time:   00:10:38
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021934/
log:    http://qa-proxy.ceph.com/teuthology/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021934/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=90a3c167ae284975940a196939724f19

    No module named 'util'



Pass
=================================================================
[5021875] rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-avl.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-bootstrap-workunit.yaml}
time:   00:11:08
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021875/

[5021880] rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-snappy.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus.yaml}
time:   00:17:20
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021880/

[5021889] rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-bootstrap-workunit.yaml}
time:   00:08:59
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021889/

[5021892] rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-snapshot-workunit.yaml}
time:   00:17:52
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021892/

[5021894] rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-lz4.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus.yaml}
time:   00:17:05
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021894/

[5021897] rbd/singleton/{all/rbd_mirror.yaml objectstore/bluestore-avl.yaml openstack.yaml supported-random-distro$/{ubuntu_latest.yaml}}
time:   00:41:11
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021897/

[5021902] rbd/valgrind/{base/install.yaml centos_latest.yaml clusters/{fixed-1.yaml openstack.yaml} objectstore/bluestore-low-osd-mem-target.yaml validator/memcheck.yaml workloads/rbd_mirror.yaml}
time:   00:53:35
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021902/

[5021903] rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-stupid.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-bootstrap-workunit.yaml}
time:   00:09:47
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021903/

[5021906] rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-avl.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-snapshot-workunit.yaml}
time:   00:20:17
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021906/

[5021908] rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus.yaml}
time:   00:17:39
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021908/

[5021911] rbd/singleton/{all/rbd_mirror.yaml objectstore/filestore-xfs.yaml openstack.yaml supported-random-distro$/{centos_8.yaml}}
time:   00:42:32
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021911/

[5021916] rbd/valgrind/{base/install.yaml centos_latest.yaml clusters/{fixed-1.yaml openstack.yaml} objectstore/bluestore-comp-zstd.yaml validator/memcheck.yaml workloads/rbd_mirror.yaml}
time:   00:53:33
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021916/

[5021917] rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-low-osd-mem-target.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-bootstrap-workunit.yaml}
time:   00:09:50
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021917/

[5021921] rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/filestore-xfs.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-snapshot-workunit.yaml}
time:   00:17:57
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021921/

[5021922] rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-avl.yaml supported-random-distro$/{centos_8.yaml} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus.yaml}
time:   00:17:01
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021922/

[5021925] rbd/singleton/{all/rbd_mirror.yaml objectstore/bluestore-stupid.yaml openstack.yaml supported-random-distro$/{ubuntu_latest.yaml}}
time:   00:41:31
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021925/

[5021931] rbd/mirror/{base/install.yaml clients/{mirror-extra.yaml mirror.yaml} cluster/{2-node.yaml openstack.yaml} msgr-failures/few.yaml objectstore/bluestore-comp-zstd.yaml supported-random-distro$/{ubuntu_latest.yaml} workloads/rbd-mirror-bootstrap-workunit.yaml}
time:   00:10:00
info:   http://pulpito.ceph.com/trociny-2020-05-04_13:22:11-rbd-master-distro-basic-smithi/5021931/
2020-05-04T15:08:23.713 INFO:teuthology.results:starting coverage generation
2020-05-04T15:08:23.726 ERROR:teuthology.results:error generating memo/results
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 35, in main
    int(args['--timeout']), args['--dry-run'])
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 84, in results
    generate_coverage(archive_dir, name)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 107, in generate_coverage
    archive_dir,
  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage': '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage'