2024-01-11T17:57:28.010 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-01-11T17:57:28.010 INFO:teuthology.results:seed: '8665'
2024-01-11T18:08:22.713 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-01-11T18:08:22.716 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2024-01-11T21:30:38.131 INFO:teuthology.results:Tests finished! gathering results...
2024-01-11T21:30:38.563 INFO:scrape:Found 6 jobs
2024-01-11T21:30:41.953 INFO:scrape:Found 1 distinct failure reasons
2024-01-11T21:30:41.954 INFO:scrape:Failure: Command failed (workunit test rbd/compare_mirror_images.sh) on smithi188 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=56122b59dff98c17a778754b7014a75c5e3a3a19 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_DEVICE_TYPE=nbd RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff RBD_MIRROR_INSTANCES=1 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/compare_mirror_images.sh'
2024-01-11T21:30:41.954 INFO:scrape:4 jobs: ['7513796', '7513795', '7513797', '7513794']
2024-01-11T21:30:41.954 INFO:scrape:suites intersection: ['clients/{mirror-extra', 'cluster/{2-node', 'conf/{disable-pool-app}', 'mirror}', 'msgr-failures/few', 'openstack}', 'rbd:mirror/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-workunit-compare-mirror-images-nbd}']
2024-01-11T21:30:41.954 INFO:scrape:suites union: ['clients/{mirror-extra', 'cluster/{2-node', 'conf/{disable-pool-app}', 'mirror}', 'msgr-failures/few', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'openstack}', 'rbd:mirror/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-workunit-compare-mirror-images-nbd}']
2024-01-11T21:30:41.954 INFO:scrape:
2024-01-11T21:30:41.955 INFO:teuthology.results:Sending results to rraja@redhat.com: Test Run: rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi
=================================================================
info:    https://pulpito.ceph.com/rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi/
failed:  4
dead:    0
running: 0
waiting: 0
queued:  0
passed:  2


Fail
=================================================================
[7513794]  rbd:mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-compare-mirror-images-nbd}
-----------------------------------------------------------------
time:   03:21:53
info:   https://pulpito.ceph.com/rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi/7513794/
log:    http://qa-proxy.ceph.com/teuthology/rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi/7513794/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a44750d672fd43fead30cc224d977bfd

    Command failed (workunit test rbd/compare_mirror_images.sh) on smithi162
    with status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=56122b59dff98c17a778754b7014a75c5e3a3a19
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_DEVICE_TYPE=nbd
    RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff
    RBD_MIRROR_INSTANCES=1 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/compare_mirror_ima
    ges.sh'

[7513795]  rbd:mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-compare-mirror-images-nbd}
-----------------------------------------------------------------
time:   00:55:04
info:   https://pulpito.ceph.com/rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi/7513795/
log:    http://qa-proxy.ceph.com/teuthology/rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi/7513795/
sentry: https://sentry.ceph.com/organizations/ceph/?query=432923f3543e471ab73f4ca7f1ab2b94

    Command failed (workunit test rbd/compare_mirror_images.sh) on smithi052
    with status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=56122b59dff98c17a778754b7014a75c5e3a3a19
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_DEVICE_TYPE=nbd
    RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff
    RBD_MIRROR_INSTANCES=1 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/compare_mirror_ima
    ges.sh'

[7513796]  rbd:mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-compare-mirror-images-nbd}
-----------------------------------------------------------------
time:   03:03:31
info:   https://pulpito.ceph.com/rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi/7513796/
log:    http://qa-proxy.ceph.com/teuthology/rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi/7513796/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e3e608d120c74311b9ad021ebc17b13d

    Command failed (workunit test rbd/compare_mirror_images.sh) on smithi188
    with status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=56122b59dff98c17a778754b7014a75c5e3a3a19
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_DEVICE_TYPE=nbd
    RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff
    RBD_MIRROR_INSTANCES=1 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/compare_mirror_ima
    ges.sh'

[7513797]  rbd:mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-compare-mirror-images-nbd}
-----------------------------------------------------------------
time:   02:24:50
info:   https://pulpito.ceph.com/rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi/7513797/
log:    http://qa-proxy.ceph.com/teuthology/rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi/7513797/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f9772acfe0e448399b48bfecb8e83f60

    Command failed (workunit test rbd/compare_mirror_images.sh) on smithi143
    with status 1: 'mkdir -p --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd --
    /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp &&
    CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=56122b59dff98c17a778754b7014a75c5e3a3a19
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1"
    CEPH_ID="mirror" PATH=$PATH:/usr/sbin
    CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror
    CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\'
    MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_DEVICE_TYPE=nbd
    RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff
    RBD_MIRROR_INSTANCES=1 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt
    u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/compare_mirror_ima
    ges.sh'



Pass
=================================================================
[7513793] rbd:mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-compare-mirror-images-nbd}
time:   03:06:30
info:   https://pulpito.ceph.com/rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi/7513793/

[7513798] rbd:mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-compare-mirror-images-nbd}
time:   02:43:16
info:   https://pulpito.ceph.com/rraja-2024-01-11_17:51:11-rbd:mirror-main-distro-default-smithi/7513798/