2023-06-21T21:40:30.536 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb
2023-06-21T21:40:30.537 INFO:teuthology.results:subset: '25/99'
2023-06-21T21:40:30.537 INFO:teuthology.results:seed: '3615'
2023-06-21T21:47:57.781 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb
2023-06-21T21:47:57.785 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-06-21T22:26:00.799 INFO:teuthology.results:Tests finished! gathering results...
2023-06-21T22:26:01.203 INFO:scrape:Found 10 jobs
2023-06-21T22:26:06.101 INFO:scrape:Found 1 distinct failure reasons
2023-06-21T22:26:06.102 INFO:scrape:Failure: Command failed (workunit test rbd/diff_continuous.sh) on smithi153 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9486e95db9be21fc8943421d32131b9edb9d2bb9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_DEVICE_TYPE=nbd adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/diff_continuous.sh'
2023-06-21T22:26:06.102 INFO:scrape:10 jobs: ['7310668', '7310669', '7310666', '7310672', '7310675', '7310671', '7310667', '7310674', '7310670', '7310673']
2023-06-21T22:26:06.103 INFO:scrape:suites intersection: ['cluster/{fixed-3', 'msgr-failures/few', 'openstack}', 'rbd/nbd/{base/install', 'thrashers/default', 'thrashosds-health', 'workloads/rbd_nbd_diff_continuous}']
2023-06-21T22:26:06.103 INFO:scrape:suites union: ['cluster/{fixed-3', 'msgr-failures/few', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-hybrid', 'openstack}', 'rbd/nbd/{base/install', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/default', 'thrashosds-health', 'workloads/rbd_nbd_diff_continuous}']
2023-06-21T22:26:06.103 INFO:scrape:
2023-06-21T22:26:06.104 INFO:teuthology.results:Sending results to idryomov@gmail.com: Test Run: dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/
failed:  10
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[7310666]  rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous}
-----------------------------------------------------------------
time:   00:22:30
info:   http://pulpito.ceph.com/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310666/
log:    http://qa-proxy.ceph.com/teuthology/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310666/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a6da472f66414be88f74857aa0e8bb76

    Command failed (workunit test rbd/diff_continuous.sh) on smithi134 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9486e95db9be21fc8943421d32131b9edb9d2bb9
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_DEVICE_TYPE=nbd adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/diff_continuous.sh'

[7310667]  rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous}
-----------------------------------------------------------------
time:   00:21:57
info:   http://pulpito.ceph.com/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310667/
log:    http://qa-proxy.ceph.com/teuthology/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310667/
sentry: https://sentry.ceph.com/organizations/ceph/?query=22d5dfdfb3d54193b0424b60a0a4eb10

    Command failed (workunit test rbd/diff_continuous.sh) on smithi172 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9486e95db9be21fc8943421d32131b9edb9d2bb9
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_DEVICE_TYPE=nbd adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/diff_continuous.sh'

[7310668]  rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous}
-----------------------------------------------------------------
time:   00:29:19
info:   http://pulpito.ceph.com/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310668/
log:    http://qa-proxy.ceph.com/teuthology/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310668/
sentry: https://sentry.ceph.com/organizations/ceph/?query=47902893de97403a994cb9dd14b628f1

    Command failed (workunit test rbd/diff_continuous.sh) on smithi153 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9486e95db9be21fc8943421d32131b9edb9d2bb9
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_DEVICE_TYPE=nbd adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/diff_continuous.sh'

[7310669]  rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous}
-----------------------------------------------------------------
time:   00:21:47
info:   http://pulpito.ceph.com/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310669/
log:    http://qa-proxy.ceph.com/teuthology/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310669/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5dd31d90e14249f58ab14e95cf2d64d0

    Command failed (workunit test rbd/diff_continuous.sh) on smithi202 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9486e95db9be21fc8943421d32131b9edb9d2bb9
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_DEVICE_TYPE=nbd adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/diff_continuous.sh'

[7310670]  rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous}
-----------------------------------------------------------------
time:   00:19:58
info:   http://pulpito.ceph.com/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310670/
log:    http://qa-proxy.ceph.com/teuthology/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310670/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c3283e89fe1c4c6193ce3bbdd8965183

    Command failed (workunit test rbd/diff_continuous.sh) on smithi133 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9486e95db9be21fc8943421d32131b9edb9d2bb9
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_DEVICE_TYPE=nbd adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/diff_continuous.sh'

[7310671]  rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous}
-----------------------------------------------------------------
time:   00:26:19
info:   http://pulpito.ceph.com/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310671/
log:    http://qa-proxy.ceph.com/teuthology/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310671/
sentry: https://sentry.ceph.com/organizations/ceph/?query=67bba95a363d44089410eb1383a5f89d

    Command failed (workunit test rbd/diff_continuous.sh) on smithi162 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9486e95db9be21fc8943421d32131b9edb9d2bb9
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_DEVICE_TYPE=nbd adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/diff_continuous.sh'

[7310672]  rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous}
-----------------------------------------------------------------
time:   00:24:31
info:   http://pulpito.ceph.com/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310672/
log:    http://qa-proxy.ceph.com/teuthology/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310672/
sentry: https://sentry.ceph.com/organizations/ceph/?query=29c71a0ee8814377885489ee02558b5a

    Command failed (workunit test rbd/diff_continuous.sh) on smithi154 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9486e95db9be21fc8943421d32131b9edb9d2bb9
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_DEVICE_TYPE=nbd adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/diff_continuous.sh'

[7310673]  rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous}
-----------------------------------------------------------------
time:   00:26:18
info:   http://pulpito.ceph.com/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310673/
log:    http://qa-proxy.ceph.com/teuthology/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310673/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a885f5656b5d42e494dbd8686eeec2d6

    Command failed (workunit test rbd/diff_continuous.sh) on smithi135 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9486e95db9be21fc8943421d32131b9edb9d2bb9
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_DEVICE_TYPE=nbd adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/diff_continuous.sh'

[7310674]  rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous}
-----------------------------------------------------------------
time:   00:29:57
info:   http://pulpito.ceph.com/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310674/
log:    http://qa-proxy.ceph.com/teuthology/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310674/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6fcc17ba0bf44ff3ae6407e30436a806

    Command failed (workunit test rbd/diff_continuous.sh) on smithi171 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9486e95db9be21fc8943421d32131b9edb9d2bb9
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_DEVICE_TYPE=nbd adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/diff_continuous.sh'

[7310675]  rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rbd_nbd_diff_continuous}
-----------------------------------------------------------------
time:   00:31:44
info:   http://pulpito.ceph.com/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310675/
log:    http://qa-proxy.ceph.com/teuthology/dis-2023-06-21_21:25:05-rbd-reef-distro-default-smithi/7310675/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ba85395411e94cf1b2a7ae6696f8393b

    Command failed (workunit test rbd/diff_continuous.sh) on smithi190 with
    status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=9486e95db9be21fc8943421d32131b9edb9d2bb9
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_DEVICE_TYPE=nbd adjust-ulimits
    ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/diff_continuous.sh'