2023-06-01T18:55:28.780 INFO:root:teuthology version: 0.0.1.dev116+g62814ab
2023-06-01T18:55:28.781 INFO:teuthology.results:seed: '831'
2023-06-01T18:56:17.292 INFO:root:teuthology version: 0.0.1.dev116+g62814ab
2023-06-01T18:56:17.294 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-06-01T19:37:20.134 INFO:teuthology.results:Tests finished! gathering results...
2023-06-01T19:37:20.332 INFO:scrape:Found 5 jobs
2023-06-01T19:37:20.811 INFO:scrape:Found 1 distinct failure reasons
2023-06-01T19:37:20.812 INFO:scrape:Failure: Command failed (workunit test rgw/test_rgw_s3_mp_reupload.sh) on smithi043 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=eec363bd667fb7172b7dbcbb4332df88236058d2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh'
2023-06-01T19:37:20.813 INFO:scrape:5 jobs: ['7293928', '7293931', '7293932', '7293929', '7293930']
2023-06-01T19:37:20.813 INFO:scrape:suites intersection: ['clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'ragweed', 'reshard', 'rgw:verify/{0-install', 's3tests-branch', 's3tests-java', 's3tests}', 'tasks/{cls']
2023-06-01T19:37:20.813 INFO:scrape:suites union: ['clusters/fixed-2', 'datacache/no_datacache', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{0-install', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-branch', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_20.04}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{cls', 'validater/lockdep}', 'validater/valgrind}']
2023-06-01T19:37:20.814 INFO:scrape:
2023-06-01T19:37:20.814 INFO:teuthology.results:Sending results to ivancich@redhat.com: Test Run: ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi/
failed:  5
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[7293928]  rgw:verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{rhel_8} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:23:31
info:   http://pulpito.ceph.com/ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293928/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293928/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2f40d712882a4cb29b1379c11b10b49a

    Command failed (workunit test rgw/test_rgw_s3_mp_reupload.sh) on smithi043
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=eec363bd667fb7172b7dbcbb4332df88236058d2
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh'

[7293929]  rgw:verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{rhel_8} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:22:34
info:   http://pulpito.ceph.com/ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293929/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293929/
sentry: https://sentry.ceph.com/organizations/ceph/?query=29d15c0952724f8289ad05a0b247d64f

    Command failed (workunit test rgw/test_rgw_s3_mp_reupload.sh) on smithi105
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=eec363bd667fb7172b7dbcbb4332df88236058d2
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh'

[7293930]  rgw:verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:22:58
info:   http://pulpito.ceph.com/ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293930/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293930/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0bdb9c4b3e3444e7b26eeb4fb5de9722

    Command failed (workunit test rgw/test_rgw_s3_mp_reupload.sh) on smithi070
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=eec363bd667fb7172b7dbcbb4332df88236058d2
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh'

[7293931]  rgw:verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_8} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:27:23
info:   http://pulpito.ceph.com/ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293931/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293931/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a9641119da364d6482a3b8299df4f473

    Command failed (workunit test rgw/test_rgw_s3_mp_reupload.sh) on smithi079
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=eec363bd667fb7172b7dbcbb4332df88236058d2
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh'

[7293932]  rgw:verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_20.04} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:23:01
info:   http://pulpito.ceph.com/ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293932/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2023-06-01_18:55:22-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293932/
sentry: https://sentry.ceph.com/organizations/ceph/?query=211262d4800a4dd19d367cc0695dd7c8

    Command failed (workunit test rgw/test_rgw_s3_mp_reupload.sh) on smithi111
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=eec363bd667fb7172b7dbcbb4332df88236058d2
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh'