2024-04-14T20:52:51.862 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-04-14T20:52:51.863 INFO:teuthology.results:subset: '0/1'
2024-04-14T20:52:51.863 INFO:teuthology.results:seed: '2462'
2024-04-15T09:48:33.329 INFO:root:teuthology version: 0.0.1.dev249+gcd45576
2024-04-15T09:48:33.359 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2024-04-15T10:20:36.034 INFO:teuthology.results:Tests finished! gathering results...
2024-04-15T10:20:39.123 INFO:scrape:Found 73 jobs
2024-04-15T10:20:40.908 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654505/teuthology.log
2024-04-15T10:20:44.830 INFO:scrape:Found 3 distinct failure reasons
2024-04-15T10:20:44.831 INFO:scrape:Failure: Command failed (workunit test cls/test_cls_user.sh) on smithi082 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'
2024-04-15T10:20:44.831 INFO:scrape:24 jobs: ['7654466', '7654519', '7654479', '7654485', '7654512', '7654503', '7654476', '7654521', '7654508', '7654489', '7654468', '7654486', '7654473', '7654471', '7654481', '7654518', '7654497', '7654500', '7654499', '7654490', '7654513', '7654470', '7654507', '7654516']
2024-04-15T10:20:44.831 INFO:scrape:suites intersection: ['cls', 'clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'ragweed', 'reshard', 'rgw/verify/{0-install', 's3tests', 's3tests-branch', 's3tests-java', 'tasks/{bucket-check', 'versioning}']
2024-04-15T10:20:44.831 INFO:scrape:suites union: ['accounts$/{main-tenant}', 'accounts$/{main}', 'accounts$/{none}', 'accounts$/{tenant}', 'cls', 'clusters/fixed-2', 'datacache/no_datacache', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'validater/valgrind}', 'versioning}']
2024-04-15T10:20:44.832 INFO:scrape:
2024-04-15T10:20:44.832 INFO:scrape:Dead: None
2024-04-15T10:20:44.832 INFO:scrape:2 jobs: ['7654505', '7654460']
2024-04-15T10:20:44.832 INFO:scrape:
2024-04-15T10:20:44.832 INFO:scrape:Failure: 
2024-04-15T10:20:44.832 INFO:scrape:2 jobs: ['7654457', '7654493']
2024-04-15T10:20:44.832 INFO:scrape:suites intersection: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'rgw/multisite/{clusters', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}']
2024-04-15T10:20:44.832 INFO:scrape:suites union: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zones', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}']
2024-04-15T10:20:44.832 INFO:scrape:
2024-04-15T10:20:44.835 INFO:teuthology.results:Sending results to ceph-qa@ceph.com: Test Run: teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi
=================================================================
info:    https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/
failed:  26
dead:    2
running: 0
waiting: 0
queued:  0
passed:  45


Fail
=================================================================
[7654457]  rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{ubuntu_latest} tasks/test_multi}
-----------------------------------------------------------------
time:   00:19:12
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654457/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654457/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ae8c51384dec4215a1c90d2cb417ca4c

[7654466]  rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
-----------------------------------------------------------------
time:   00:17:50
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654466/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654466/
sentry: https://sentry.ceph.com/organizations/ceph/?query=c5caaa057b5645dfae9ef614f004fff3

    Command failed (workunit test cls/test_cls_user.sh) on smithi082 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654468]  rgw/verify/{0-install accounts$/{main} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:24:04
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654468/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654468/
sentry: https://sentry.ceph.com/organizations/ceph/?query=837adc131f6a4d098df23fbacf06b71a

    Command failed (workunit test cls/test_cls_user.sh) on smithi098 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654470]  rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
-----------------------------------------------------------------
time:   00:21:54
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654470/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654470/
sentry: https://sentry.ceph.com/organizations/ceph/?query=699eca2eacbe43adabc221a37fee8d60

    Command failed (workunit test cls/test_cls_user.sh) on smithi016 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654471]  rgw/verify/{0-install accounts$/{none} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:24:10
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654471/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654471/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9841a195a7da4c9385241214693d5d0d

    Command failed (workunit test cls/test_cls_user.sh) on smithi032 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654473]  rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
-----------------------------------------------------------------
time:   00:21:44
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654473/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654473/
sentry: https://sentry.ceph.com/organizations/ceph/?query=31f2cb2f134d46b49a51cbf931194289

    Command failed (workunit test cls/test_cls_user.sh) on smithi066 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654476]  rgw/verify/{0-install accounts$/{none} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:19:38
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654476/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654476/
sentry: https://sentry.ceph.com/organizations/ceph/?query=25f931a48cef42778e08e6dc849905e2

    Command failed (workunit test cls/test_cls_user.sh) on smithi047 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654479]  rgw/verify/{0-install accounts$/{none} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
-----------------------------------------------------------------
time:   00:17:52
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654479/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654479/
sentry: https://sentry.ceph.com/organizations/ceph/?query=47f523ffeb4a4ea5a472cf2c7281d129

    Command failed (workunit test cls/test_cls_user.sh) on smithi018 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654481]  rgw/verify/{0-install accounts$/{main} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:23:30
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654481/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654481/
sentry: https://sentry.ceph.com/organizations/ceph/?query=02920bec2a5044a9a53f7795247bdec3

    Command failed (workunit test cls/test_cls_user.sh) on smithi033 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654485]  rgw/verify/{0-install accounts$/{main} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
-----------------------------------------------------------------
time:   00:17:17
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654485/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654485/
sentry: https://sentry.ceph.com/organizations/ceph/?query=86101ee2ca144746a784d9a1800607e6

    Command failed (workunit test cls/test_cls_user.sh) on smithi043 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654486]  rgw/verify/{0-install accounts$/{none} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:22:52
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654486/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654486/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e8682e4c52db418fa2c262aa48f38c7e

    Command failed (workunit test cls/test_cls_user.sh) on smithi049 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654489]  rgw/verify/{0-install accounts$/{main} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
-----------------------------------------------------------------
time:   00:21:16
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654489/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654489/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ea4754b000584ceda9da5968c81ec056

    Command failed (workunit test cls/test_cls_user.sh) on smithi069 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654490]  rgw/verify/{0-install accounts$/{tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:23:01
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654490/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654490/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0fafc833603a492d924975107f3f8e32

    Command failed (workunit test cls/test_cls_user.sh) on smithi116 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654493]  rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{ubuntu_latest} tasks/test_multi}
-----------------------------------------------------------------
time:   00:19:39
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654493/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654493/
sentry: https://sentry.ceph.com/organizations/ceph/?query=5a1401259f7b4fb7b1cf9cc3a631494f

[7654497]  rgw/verify/{0-install accounts$/{none} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
-----------------------------------------------------------------
time:   00:23:06
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654497/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654497/
sentry: https://sentry.ceph.com/organizations/ceph/?query=15b25aca5b3c4478acd89f20780bd175

    Command failed (workunit test cls/test_cls_user.sh) on smithi059 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654499]  rgw/verify/{0-install accounts$/{main} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:17:31
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654499/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654499/
sentry: https://sentry.ceph.com/organizations/ceph/?query=86586d986fb54469a7960a055f232428

    Command failed (workunit test cls/test_cls_user.sh) on smithi115 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654500]  rgw/verify/{0-install accounts$/{tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
-----------------------------------------------------------------
time:   00:22:39
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654500/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654500/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6fc6da7a7f9b4be18aeb6a34f3cdf0b3

    Command failed (workunit test cls/test_cls_user.sh) on smithi105 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654503]  rgw/verify/{0-install accounts$/{none} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:18:01
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654503/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654503/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b18ed8b687fb4d25a353789fb1b57d95

    Command failed (workunit test cls/test_cls_user.sh) on smithi037 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654507]  rgw/verify/{0-install accounts$/{main} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
-----------------------------------------------------------------
time:   00:18:13
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654507/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654507/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e36844baed7b45e3bfb554b99607f88e

    Command failed (workunit test cls/test_cls_user.sh) on smithi134 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654508]  rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:22:15
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654508/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654508/
sentry: https://sentry.ceph.com/organizations/ceph/?query=aa23059fe6874fb19236bc0a5fb88a9a

    Command failed (workunit test cls/test_cls_user.sh) on smithi122 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654512]  rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
-----------------------------------------------------------------
time:   00:22:28
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654512/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654512/
sentry: https://sentry.ceph.com/organizations/ceph/?query=982be4b93f8b4393be84d64f24affebc

    Command failed (workunit test cls/test_cls_user.sh) on smithi078 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654513]  rgw/verify/{0-install accounts$/{main} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:21:46
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654513/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654513/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e0fefb4e34a54b00bb373b74c96097d1

    Command failed (workunit test cls/test_cls_user.sh) on smithi104 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654516]  rgw/verify/{0-install accounts$/{tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
-----------------------------------------------------------------
time:   00:18:26
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654516/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654516/
sentry: https://sentry.ceph.com/organizations/ceph/?query=74949b6b55fe4da2ab41d4a6a1ee29be

    Command failed (workunit test cls/test_cls_user.sh) on smithi139 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654518]  rgw/verify/{0-install accounts$/{none} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:19:27
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654518/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654518/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8fee4a55977949aa9acfe045aae4d639

    Command failed (workunit test cls/test_cls_user.sh) on smithi118 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654519]  rgw/verify/{0-install accounts$/{main} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
-----------------------------------------------------------------
time:   00:21:01
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654519/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654519/
sentry: https://sentry.ceph.com/organizations/ceph/?query=634a852e13894004982fae2dbeb1ae86

    Command failed (workunit test cls/test_cls_user.sh) on smithi159 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'

[7654521]  rgw/verify/{0-install accounts$/{main-tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:22:04
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654521/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654521/
sentry: https://sentry.ceph.com/organizations/ceph/?query=25305427a63f49fdb7b4345df60b2e45

    Command failed (workunit test cls/test_cls_user.sh) on smithi007 with
    status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1
    CEPH_REF=c2d8c3155d854ee204cb7c7261232571229d1d54
    TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0"
    PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0
    CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0
    CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage timeout 3h
    /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh'



Dead
=================================================================
[7654460]  rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654460/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654460/

    Error reimaging machines: Failed to power on smithi155

[7654505]  rgw/notifications/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides tasks/others/{0-install supported-distros/{ubuntu_latest} test_others}}
-----------------------------------------------------------------
time:   00:00:00
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654505/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654505/

    Error reimaging machines: Failed to power on smithi131



Pass
=================================================================
[7654449] rgw/cloud-transition/{cluster ignore-pg-availability overrides s3tests-branch supported-random-distro$/{centos_latest} tasks/cloud_transition_s3tests}
time:   00:27:04
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654449/

[7654450] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest}
time:   00:30:49
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654450/

[7654451] rgw/d4n/{cluster ignore-pg-availability overrides supported-random-distro$/{centos_latest} tasks/rgw_d4ntests}
time:   00:12:44
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654451/

[7654452] rgw/dbstore/{cluster ignore-pg-availability overrides s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/rgw_s3tests}
time:   00:17:25
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654452/

[7654453] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default ignore-pg-availability overrides s3a-hadoop supported-random-distro$/{ubuntu_latest}}
time:   00:18:34
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654453/

[7654454] rgw/lifecycle/{cluster ignore-pg-availability overrides s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/rgw_s3tests}
time:   00:39:32
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654454/

[7654455] rgw/lua/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides supported-distros/{ubuntu_latest} tasks/{0-install test_lua}}
time:   00:14:23
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654455/

[7654456] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_bucket_quota ubuntu_latest}
time:   00:14:52
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654456/

[7654458] rgw/notifications/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides tasks/amqp/{0-install centos_latest test_amqp}}
time:   00:16:05
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654458/

[7654459] rgw/service-token/{clusters/fixed-1 frontend/beast ignore-pg-availability overrides tasks/service-token ubuntu_latest}
time:   00:13:26
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654459/

[7654461] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast s3tests-branch supported-random-distro$/{centos_latest} tasks/{0-install 1-keycloak 2-s3tests}}
time:   00:33:47
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654461/

[7654462] rgw/tempest/{0-install clusters/fixed-1 frontend/beast ignore-pg-availability overrides s3tests-branch tasks/s3tests ubuntu_latest}
time:   00:32:34
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654462/

[7654463] rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota}
time:   00:15:13
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654463/

[7654464] rgw/tools/{centos_latest cluster ignore-pg-availability tasks}
time:   02:37:29
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654464/

[7654465] rgw/upgrade/{1-install/quincy/{distro$/{ubuntu_latest} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides}
time:   00:16:02
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654465/

[7654467] rgw/website/{clusters/fixed-2 frontend/beast http ignore-pg-availability overrides s3tests-branch tasks/s3tests-website ubuntu_latest}
time:   00:21:35
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654467/

[7654469] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_multipart_upload ubuntu_latest}
time:   00:14:05
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654469/

[7654472] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:15:13
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654472/

[7654474] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest}
time:   00:21:31
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654474/

[7654475] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:23:02
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654475/

[7654477] rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload}
time:   00:15:08
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654477/

[7654478] rgw/upgrade/{1-install/reef/{distro$/{centos_latest} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides}
time:   00:14:38
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654478/

[7654480] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_user_quota ubuntu_latest}
time:   00:13:07
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654480/

[7654482] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_bucket_quota ubuntu_latest}
time:   00:15:10
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654482/

[7654483] rgw/notifications/{beast bluestore-bitmap fixed-2 ignore-pg-availability overrides tasks/kafka/{0-install supported-distros/{ubuntu_latest} test_kafka}}
time:   00:24:16
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654483/

[7654484] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}}
time:   00:12:42
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654484/

[7654487] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest}
time:   00:21:51
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654487/

[7654488] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_multipart_upload ubuntu_latest}
time:   00:13:56
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654488/

[7654491] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:15:03
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654491/

[7654492] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 ignore-pg-availability overrides s3a-hadoop supported-random-distro$/{ubuntu_latest}}
time:   00:18:39
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654492/

[7654494] rgw/tempest/{0-install clusters/fixed-1 frontend/beast ignore-pg-availability overrides s3tests-branch tasks/tempest ubuntu_latest}
time:   00:21:46
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654494/

[7654495] rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests}
time:   00:23:37
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654495/

[7654496] rgw/upgrade/{1-install/reef/{distro$/{ubuntu_latest} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides}
time:   00:16:00
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654496/

[7654498] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:21:57
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654498/

[7654501] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest}
time:   00:21:48
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654501/

[7654502] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_user_quota ubuntu_latest}
time:   00:14:11
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654502/

[7654504] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_bucket_quota ubuntu_latest}
time:   00:14:44
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654504/

[7654506] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}}
time:   00:14:39
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654506/

[7654509] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_multipart_upload ubuntu_latest}
time:   00:14:15
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654509/

[7654510] rgw/thrash/{clusters/fixed-2 frontend/beast ignore-pg-availability install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota}
time:   00:15:06
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654510/

[7654511] rgw/upgrade/{1-install/quincy/{distro$/{ubuntu_latest} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides}
time:   00:16:14
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654511/

[7654514] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch ubuntu_latest}
time:   00:21:56
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654514/

[7654515] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:15:27
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654515/

[7654517] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:22:17
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654517/

[7654520] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_user_quota ubuntu_latest}
time:   00:14:04
info:   https://pulpito.ceph.com/teuthology-2024-04-12_20:40:14-rgw-main-distro-default-smithi/7654520/