2023-06-01T16:55:56.299 INFO:root:teuthology version: 0.0.1.dev116+g62814ab
2023-06-01T16:55:56.300 INFO:teuthology.results:seed: '831'
2023-06-01T16:56:43.992 INFO:root:teuthology version: 0.0.1.dev116+g62814ab
2023-06-01T16:56:43.996 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-06-01T17:30:46.109 INFO:teuthology.results:Tests finished! gathering results...
2023-06-01T17:30:46.304 INFO:scrape:Found 5 jobs
2023-06-01T17:30:46.751 INFO:scrape:Found 1 distinct failure reasons
2023-06-01T17:30:46.752 INFO:scrape:Failure: Command failed (workunit test rgw/test_rgw_s3_mp_reupload.sh) on smithi097 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=be3b294015607d284fc818458893e49e08d6146c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh'
2023-06-01T17:30:46.752 INFO:scrape:5 jobs: ['7293861', '7293862', '7293863', '7293864', '7293865']
2023-06-01T17:30:46.753 INFO:scrape:suites intersection: ['clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'ragweed', 'reshard', 'rgw:verify/{0-install', 's3tests-branch', 's3tests-java', 's3tests}', 'tasks/{cls']
2023-06-01T17:30:46.753 INFO:scrape:suites union: ['clusters/fixed-2', 'datacache/no_datacache', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw:verify/{0-install', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-branch', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_20.04}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{cls', 'validater/lockdep}', 'validater/valgrind}']
2023-06-01T17:30:46.753 INFO:scrape:
2023-06-01T17:30:46.754 INFO:teuthology.results:Sending results to ivancich@redhat.com: Test Run: ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi/
failed:  5
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[7293861]  rgw:verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{rhel_8} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:23:18
info:   http://pulpito.ceph.com/ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293861/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293861/
sentry: https://sentry.ceph.com/organizations/ceph/?query=13e1d981e3b142f89e27b092c111fa17

    Command failed (workunit test rgw/test_rgw_s3_mp_reupload.sh) on smithi097
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=be3b294015607d284fc818458893e49e08d6146c
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh'

[7293862]  rgw:verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{rhel_8} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:26:05
info:   http://pulpito.ceph.com/ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293862/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293862/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bced2a15930c4fbb83bf5357de4e476c

    Command failed (workunit test rgw/test_rgw_s3_mp_reupload.sh) on smithi160
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=be3b294015607d284fc818458893e49e08d6146c
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh'

[7293863]  rgw:verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:24:04
info:   http://pulpito.ceph.com/ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293863/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293863/
sentry: https://sentry.ceph.com/organizations/ceph/?query=21d0fd7474534926ac673bd04ec9339f

    Command failed (workunit test rgw/test_rgw_s3_mp_reupload.sh) on smithi172
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=be3b294015607d284fc818458893e49e08d6146c
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh'

[7293864]  rgw:verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_8} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   00:21:07
info:   http://pulpito.ceph.com/ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293864/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293864/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c768aa6b233c4875995f302206058681

    Command failed (workunit test rgw/test_rgw_s3_mp_reupload.sh) on smithi053
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=be3b294015607d284fc818458893e49e08d6146c
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh'

[7293865]  rgw:verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_20.04} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests} validater/lockdep}
-----------------------------------------------------------------
time:   00:22:17
info:   http://pulpito.ceph.com/ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293865/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2023-06-01_16:55:42-rgw:verify-wip-add-reupload-testing-distro-default-smithi/7293865/
sentry: https://sentry.ceph.com/organizations/ceph/?query=83571ea6977445539c07e3765cbc9e78

    Command failed (workunit test rgw/test_rgw_s3_mp_reupload.sh) on smithi121
    with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd
    -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=be3b294015607d284fc818458893e49e08d6146c
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo
    ne.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh'