2021-08-19T04:42:18.424 INFO:root:teuthology version: 1.1.0-321319b1
2021-08-19T04:42:18.425 INFO:teuthology.results:subset: '1/3'
2021-08-19T04:42:18.426 INFO:teuthology.results:seed: '831'
2021-08-19T13:12:53.072 INFO:root:teuthology version: 1.1.0-321319b1
2021-08-19T13:12:53.120 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-08-19T14:16:57.267 INFO:teuthology.results:Tests finished! gathering results...
2021-08-19T14:16:58.219 INFO:scrape:Found 16 jobs
2021-08-19T14:16:59.782 INFO:scrape:Found 1 distinct failure reasons
2021-08-19T14:16:59.783 INFO:scrape:Failure: Command failed (workunit test cls/test_cls_rgw.sh) on gibba031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=94331eed21bb7bf80329b82f88198273fc21f366 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh'
2021-08-19T14:16:59.783 INFO:scrape:8 jobs: ['6346602', '6346596', '6346591', '6346603', '6346592', '6346589', '6346601', '6346598']
2021-08-19T14:16:59.784 INFO:scrape:suites intersection: ['centos_latest', 'clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw/verify/{0-install', 's3tests-java', 's3tests}', 'tasks/{cls']
2021-08-19T14:16:59.784 INFO:scrape:suites union: ['centos_latest', 'clusters/fixed-2', 'datacache/no_datacache', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'tasks/{cls', 'validater/lockdep}', 'validater/valgrind}']
2021-08-19T14:16:59.784 INFO:scrape:
2021-08-19T14:16:59.785 INFO:teuthology.results:Sending results to ivancich@redhat.com: Test Run: ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba
=================================================================
info:    http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/
logs:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/
failed:  8
dead:    0
running: 0
waiting: 0
queued:  0
passed:  8


Fail
=================================================================
[6346589]  rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:12:42
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346589/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346589/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1b81a8d5a2ef4bcd89baac724b6a9c53

    Command failed (workunit test cls/test_cls_rgw.sh) on gibba020 with status
    1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=94331eed21bb7bf80329b82f88198273fc21f366
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh'

[6346591]  rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:14:24
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346591/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346591/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6ae6b96535b248dca45e54c562d788fe

    Command failed (workunit test cls/test_cls_rgw.sh) on gibba020 with status
    1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=94331eed21bb7bf80329b82f88198273fc21f366
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh'

[6346592]  rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:13:59
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346592/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346592/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b917bcdc74504d9a9608e83fa267f696

    Command failed (workunit test cls/test_cls_rgw.sh) on gibba028 with status
    1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=94331eed21bb7bf80329b82f88198273fc21f366
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh'

[6346596]  rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:14:36
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346596/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346596/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c927cdc83fd241db860e35172871189d

    Command failed (workunit test cls/test_cls_rgw.sh) on gibba028 with status
    1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=94331eed21bb7bf80329b82f88198273fc21f366
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh'

[6346598]  rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:13:24
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346598/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346598/
sentry: https://sentry.ceph.com/organizations/ceph/?query=332cbd143f98415e923b268791a46d80

    Command failed (workunit test cls/test_cls_rgw.sh) on gibba008 with status
    1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=94331eed21bb7bf80329b82f88198273fc21f366
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh'

[6346601]  rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:13:12
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346601/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346601/
sentry: https://sentry.ceph.com/organizations/ceph/?query=49392c4f2e094a7ba0dd553392dfedaa

    Command failed (workunit test cls/test_cls_rgw.sh) on gibba008 with status
    1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=94331eed21bb7bf80329b82f88198273fc21f366
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh'

[6346602]  rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:13:12
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346602/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346602/
sentry: https://sentry.ceph.com/organizations/ceph/?query=7883f2341d5b4fb49e3b83ed92bbbf29

    Command failed (workunit test cls/test_cls_rgw.sh) on gibba031 with status
    1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=94331eed21bb7bf80329b82f88198273fc21f366
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh'

[6346603]  rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:13:28
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346603/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346603/
sentry: https://sentry.ceph.com/organizations/ceph/?query=38e678317e754d9fb6b585b87a5540f0

    Command failed (workunit test cls/test_cls_rgw.sh) on gibba011 with status
    1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=94331eed21bb7bf80329b82f88198273fc21f366
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh'



Pass
=================================================================
[6346590] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:06:16
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346590/

[6346593] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:24:27
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346593/

[6346594] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   00:58:42
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346594/

[6346595] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:23:11
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346595/

[6346597] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:27:04
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346597/

[6346599] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:23:45
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346599/

[6346600] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:04:12
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346600/

[6346604] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   00:56:00
info:   http://pulpito.ceph.com/ivancich-2021-08-19_04:39:03-rgw-wip-fix-bi-test-error-distro-basic-gibba/6346604/