2023-11-24T20:06:09.271 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-24T20:06:09.271 INFO:teuthology.results:seed: '124' 2023-11-24T20:06:53.282 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-24T20:06:53.286 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-11-24T22:57:05.095 INFO:teuthology.results:Tests finished! gathering results... 2023-11-24T22:57:05.494 INFO:scrape:Found 4 jobs 2023-11-24T22:57:05.819 INFO:scrape:Found 2 distinct failure reasons 2023-11-24T22:57:05.820 INFO:scrape:Failure: Command failed (workunit test rbd/mirror_compare2.sh) on smithi172 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e4f650c2ad153911db201c5e75790fcd6b44823f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_DEVICE_TYPE=nbd RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff RBD_MIRROR_INSTANCES=1 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/mirror_compare2.sh' 2023-11-24T22:57:05.820 INFO:scrape:1 jobs: ['7465792'] 2023-11-24T22:57:05.820 INFO:scrape:suites: ['clients/{mirror-extra', 'cluster/{2-node', 'conf/{disable-pool-app}', 'mirror}', 'msgr-failures/few', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rbd:mirror/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-workunit-mirror-compare2-nbd}'] 2023-11-24T22:57:05.820 INFO:scrape: 2023-11-24T22:57:05.820 INFO:scrape:Failure: Command failed (workunit test rbd/mirror_compare.sh) on smithi141 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e4f650c2ad153911db201c5e75790fcd6b44823f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_DEVICE_TYPE=nbd RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff RBD_MIRROR_INSTANCES=1 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/mirror_compare.sh' 2023-11-24T22:57:05.820 INFO:scrape:1 jobs: ['7465790'] 2023-11-24T22:57:05.820 INFO:scrape:suites: ['clients/{mirror-extra', 'cluster/{2-node', 'conf/{disable-pool-app}', 'mirror}', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'openstack}', 'rbd:mirror/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-snapshot-workunit-mirror-compare-nbd}'] 2023-11-24T22:57:05.821 INFO:scrape: 2023-11-24T22:57:05.821 INFO:teuthology.results:Sending results to rraja@redhat.com: Test Run: rraja-2023-11-24_20:06:04-rbd:mirror-wip-ajarr-61617-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/rraja-2023-11-24_20:06:04-rbd:mirror-wip-ajarr-61617-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/rraja-2023-11-24_20:06:04-rbd:mirror-wip-ajarr-61617-distro-default-smithi/ failed: 2 dead: 0 running: 0 waiting: 0 queued: 0 passed: 2 Fail ================================================================= [7465790] rbd:mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-mirror-compare-nbd} ----------------------------------------------------------------- time: 00:13:25 info: https://pulpito.ceph.com/rraja-2023-11-24_20:06:04-rbd:mirror-wip-ajarr-61617-distro-default-smithi/7465790/ log: http://qa-proxy.ceph.com/teuthology/rraja-2023-11-24_20:06:04-rbd:mirror-wip-ajarr-61617-distro-default-smithi/7465790/ sentry: https://sentry.ceph.com/organizations/ceph/?query=67ecbce49c284157ab6fd00de1c89861 Command failed (workunit test rbd/mirror_compare.sh) on smithi141 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e4f650c2ad153911db201c5e75790fcd6b44823f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_DEVICE_TYPE=nbd RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff RBD_MIRROR_INSTANCES=1 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/mirror_compare.sh' [7465792] rbd:mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-workunit-mirror-compare2-nbd} ----------------------------------------------------------------- time: 00:13:11 info: https://pulpito.ceph.com/rraja-2023-11-24_20:06:04-rbd:mirror-wip-ajarr-61617-distro-default-smithi/7465792/ log: http://qa-proxy.ceph.com/teuthology/rraja-2023-11-24_20:06:04-rbd:mirror-wip-ajarr-61617-distro-default-smithi/7465792/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d3fb29e32e9e450f91ee9efbb80a90fd Command failed (workunit test rbd/mirror_compare2.sh) on smithi172 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e4f650c2ad153911db201c5e75790fcd6b44823f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS=\'\' MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_DEVICE_TYPE=nbd RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff RBD_MIRROR_INSTANCES=1 RBD_MIRROR_USE_EXISTING_CLUSTER=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubunt u/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/mirror_compare2.sh ' Pass ================================================================= [7465789] rbd:mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_latest} workloads/rbd-mirror-snapshot-workunit-mirror-compare-krbd} time: 01:33:25 info: https://pulpito.ceph.com/rraja-2023-11-24_20:06:04-rbd:mirror-wip-ajarr-61617-distro-default-smithi/7465789/ [7465791] rbd:mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} workloads/rbd-mirror-snapshot-workunit-mirror-compare2-krbd} time: 02:41:12 info: https://pulpito.ceph.com/rraja-2023-11-24_20:06:04-rbd:mirror-wip-ajarr-61617-distro-default-smithi/7465791/