2021-05-06T15:49:35.003 INFO:root:teuthology version: 1.1.0-19220a3b
2021-05-06T15:49:35.004 INFO:teuthology.results:subset: '1/20'
2021-05-06T15:49:35.004 INFO:teuthology.results:seed: '831'
2021-05-06T16:41:49.117 INFO:root:teuthology version: 1.1.0-19220a3b
2021-05-06T16:41:49.161 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2021-05-06T18:07:55.403 INFO:teuthology.results:Tests finished! gathering results...
2021-05-06T18:07:58.892 INFO:scrape:Found 38 jobs
2021-05-06T18:08:03.146 INFO:scrape:Found 4 distinct failure reasons
2021-05-06T18:08:03.147 INFO:scrape:Failure: Command failed on gibba006 with status 17: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 5762561765 --totp-seed KUS2OWMPPOL267I6PE5HWUEKEAW6YT2V66RXIXU7USTYIVA4ENBPBVVCNTYJ3ALA --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'
2021-05-06T18:08:03.147 INFO:scrape:17 jobs: ['6101182', '6101170', '6101191', '6101175', '6101189', '6101174', '6101173', '6101158', '6101157', '6101187', '6101185', '6101192', '6101162', '6101163', '6101169', '6101178', '6101167']
2021-05-06T18:08:03.148 INFO:scrape:suites intersection: []
2021-05-06T18:08:03.148 INFO:scrape:suites union: ['1-ceph-install/install', '2-kms/barbican', '2-kms/kmip', '2-kms/testing', '2-kms/vault_kv', '2-kms/vault_old', '2-kms/vault_transit', '3-rgw/rgw', '4-tests/{s3tests}', 'bluestore-bitmap', 'clusters/fixed-2', 'fixed-2', 'frontend/beast', 'frontend/civetweb', 'http', 'ignore-pg-availability', 'install', 'objectstore', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'pool-type', 'rgw/crypt/{0-cluster/fixed-1', 'rgw/multifs/{clusters/fixed-2', 'rgw/notifications/{beast', 'rgw/singleton/{all/radosgw-admin', 'rgw/sts/{cluster', 'rgw/thrash/{civetweb', 'rgw/website/{clusters/fixed-2', 'rgw_frontend/beast', 'rgw_frontend/civetweb', 'rgw_pool_type/ec', 'rgw_pool_type/replicated', 'ststests', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{rhel_8}}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'tasks/rgw_ragweed', 'tasks/rgw_s3tests', 'tasks/s3tests-website', 'tasks/{0-install', 'test_kafka}}', 'thrasher/default', 'thrashosds-health', 'ubuntu_latest', 'ubuntu_latest}', 'webidentity}}', 'workload/rgw_s3tests}']
2021-05-06T18:08:03.149 INFO:scrape:
2021-05-06T18:08:03.149 INFO:scrape:Failure: rgw multisite test failures
2021-05-06T18:08:03.150 INFO:scrape:2 jobs: ['6101181', '6101172']
2021-05-06T18:08:03.150 INFO:scrape:suites intersection: ['frontend/civetweb', 'ignore-pg-availability', 'omap_limits', 'overrides', 'rgw/multisite/{clusters', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}']
2021-05-06T18:08:03.151 INFO:scrape:suites union: ['frontend/civetweb', 'ignore-pg-availability', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}']
2021-05-06T18:08:03.151 INFO:scrape:
2021-05-06T18:08:03.152 INFO:scrape:Failure: Command failed on gibba006 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test_account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|.*test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_services_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*test_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticweb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.*test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds'"
2021-05-06T18:08:03.152 INFO:scrape:1 jobs: ['6101164']
2021-05-06T18:08:03.153 INFO:scrape:suites: ['frontend/beast', 'overrides', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}']
2021-05-06T18:08:03.153 INFO:scrape:
2021-05-06T18:08:03.154 INFO:scrape:Failure: too many values to unpack (expected 1)
2021-05-06T18:08:03.154 INFO:scrape:2 jobs: ['6101171', '6101177']
2021-05-06T18:08:03.154 INFO:scrape:suites intersection: ['frontend/civetweb', 'overrides', 'rgw/singleton/{all/rgw_cephadm']
2021-05-06T18:08:03.155 INFO:scrape:suites union: ['frontend/civetweb', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'rgw/singleton/{all/rgw_cephadm', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 'supported-random-distro$/{rhel_8}}', 'supported-random-distro$/{ubuntu_latest}}']
2021-05-06T18:08:03.155 INFO:scrape:
2021-05-06T18:08:03.157 INFO:teuthology.results:Sending results to ivancich@redhat.com: Test Run: ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba
=================================================================
info:    http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/
logs:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/
failed:  22
dead:    0
running: 0
waiting: 0
queued:  0
passed:  16


Fail
=================================================================
[6101157]  rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:06:28
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101157/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101157/
sentry: https://sentry.ceph.com/organizations/ceph/?query=b21ec42791e4436884004f2325f3f6c3

    Command failed on gibba045 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr
    --format json -n client.0 --cluster ceph user suspend --uid foo'

[6101158]  rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:21:56
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101158/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101158/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3f8c2583196d4aea8ec9ebb3056e480c

    Command failed on gibba035 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 0598985148 --totp-seed
    CDEPNY3HE3XGE6HYU2YQD4G635EKSEN2ZL6PNUSP5HU7Y3BAPHJLH5VAOO5RVPE6 --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'

[6101162]  rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-random-distro$/{ubuntu_latest} tasks/{0-install test_kafka}}
-----------------------------------------------------------------
time:   00:18:40
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101162/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101162/
sentry: https://sentry.ceph.com/organizations/ceph/?query=93783277e0e8440aaacc2f018d49770d

    Command failed on gibba009 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm
    --uid foo.client.0 --purge-data --cluster ceph'

[6101163]  rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast supported-random-distro$/{centos_8} tasks/{0-install ststests webidentity}}
-----------------------------------------------------------------
time:   00:06:53
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101163/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101163/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2c972c95a4a345b2b84b2b8b207534a8

    Command failed on gibba031 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 7166460444 --totp-seed
    N757W47WHNY6DM2FRKC3RGK3GJBRB6LY2ZBDO5KVY25NWRCK5547OBX56255K6TZ --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'

[6101164]  rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest}
-----------------------------------------------------------------
time:   00:19:19
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101164/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101164/
sentry: https://sentry.ceph.com/organizations/ceph/?query=20c9cecce4db4dc2bb82374d84972b60

    Command failed on gibba006 with status 1: "cd /home/ubuntu/cephtest/tempest
    && source .tox/venv/bin/activate && tempest run --workspace-path
    /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex
    '^tempest.api.object_storage' --black-regex '.*test_account_quotas_negative
    .AccountQuotasNegativeTest.test_user_modify_quota|.*test_container_acl_nega
    tive.ObjectACLsNegativeTest.*|.*test_container_services_negative.ContainerN
    egativeTest.test_create_container_metadata_.*|.*test_container_staticweb.St
    aticWebTest.test_web_index|.*test_container_staticweb.StaticWebTest.test_we
    b_listing_css|.*test_container_synchronization.*|.*test_object_services.Pub
    licObjectTest.test_access_public_container_object_without_using_creds'"

[6101167]  rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest}
-----------------------------------------------------------------
time:   00:11:39
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101167/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101167/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ec4c7354c0ff4a389440f11926b28a8a

    Command failed on gibba008 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 7285017854 --totp-seed
    54GTOUI3B4OZJKGHJHNK33RX3GNANCVPK3VZFCRI2IDZZ4VVVCEF2LNI7PQHDZZ6 --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'

[6101169]  rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:06:59
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101169/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101169/
sentry: https://sentry.ceph.com/organizations/ceph/?query=15e86d5c60d9424a953465b64d7cd694

    Command failed on gibba025 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 4928798228 --totp-seed
    F5DNTAHPLM2GHQR5L62HBOUNLWKOB7E5JD652VFXPRQ67RVQLF7T4VLKXI5DK2J5 --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'

[6101170]  rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_ragweed ubuntu_latest}
-----------------------------------------------------------------
time:   00:06:29
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101170/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101170/
sentry: https://sentry.ceph.com/organizations/ceph/?query=ef33b3989c0042ba882782709e73a087

    Command failed on gibba014 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm
    --uid ragweed.client.0 --purge-data'

[6101171]  rgw/singleton/{all/rgw_cephadm frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101171/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101171/
sentry: https://sentry.ceph.com/organizations/ceph/?query=81971705d80343719267435a26690389

    too many values to unpack (expected 1)

[6101172]  rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/two-zonegroup supported-random-distro$/{ubuntu_latest} tasks/test_multi}
-----------------------------------------------------------------
time:   00:20:20
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101172/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101172/
sentry: https://sentry.ceph.com/organizations/ceph/?query=f4524eb390a04ecf84e101de60b1c90b

    rgw multisite test failures

[6101173]  rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests}
-----------------------------------------------------------------
time:   00:10:15
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101173/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101173/
sentry: https://sentry.ceph.com/organizations/ceph/?query=931a746fb26c4c1bab8c5878745c1564

    Command failed on gibba011 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 9213832907 --totp-seed
    R7BQ52UU3LAKZC3WRMP2HCTGBRHYC34GAJUVQYETZJYHGKFMFLAATGBUBZGSXIUI --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'

[6101174]  rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:06:25
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101174/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101174/
sentry: https://sentry.ceph.com/organizations/ceph/?query=957eed5281064a4493c06c4d51da2b8e

    Command failed on gibba025 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 8524068224 --totp-seed
    RDA2W4ZWPIKE3ERZGZELXA73LJCPAOAT7QTSZ7ZMHDTHYXWIESVIOHBEL2FR335T --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'

[6101175]  rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_s3tests ubuntu_latest}
-----------------------------------------------------------------
time:   00:06:25
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101175/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101175/
sentry: https://sentry.ceph.com/organizations/ceph/?query=4d1a08ad17da4963a396a939217919f1

    Command failed on gibba002 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 9095550902 --totp-seed
    H2MR3UCS5AVBJZEIEKCNG4KBWLWWWA7Z35LSX5B5MP4PEOFDHIG6HJH76J6HOMN7 --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'

[6101177]  rgw/singleton/{all/rgw_cephadm frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101177/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101177/
sentry: https://sentry.ceph.com/organizations/ceph/?query=62027ae36f3a4b39b7f1adae0c7b4ed0

    too many values to unpack (expected 1)

[6101178]  rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:05:55
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101178/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101178/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d2a0be86bb474c9281c452516b46c774

    Command failed on gibba030 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 8830693433 --totp-seed
    WCRL6WSPQETJY6SG64N47FCETHLCZFHYMBQ3FJW6UV7YH4ZTMMQJLXVMWNQXWF6R --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'

[6101181]  rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{ubuntu_latest} tasks/test_multi}
-----------------------------------------------------------------
time:   00:28:06
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101181/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101181/
sentry: https://sentry.ceph.com/organizations/ceph/?query=3ed63610955241f1a79442c847ed7600

    rgw multisite test failures

[6101182]  rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/civetweb supported-random-distro$/{rhel_8} tasks/{0-install ststests webidentity}}
-----------------------------------------------------------------
time:   00:13:15
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101182/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101182/
sentry: https://sentry.ceph.com/organizations/ceph/?query=02d6b2017cbf4587b78ff1ab19ee7c93

    Command failed on gibba006 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 5762561765 --totp-seed
    KUS2OWMPPOL267I6PE5HWUEKEAW6YT2V66RXIXU7USTYIVA4ENBPBVVCNTYJ3ALA --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'

[6101185]  rgw/website/{clusters/fixed-2 frontend/civetweb http overrides tasks/s3tests-website ubuntu_latest}
-----------------------------------------------------------------
time:   00:11:25
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101185/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101185/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a5c92c11b7044f8583a7d5169716ff4a

    Command failed on gibba025 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 1904599258 --totp-seed
    2FXLOP3RWTGILQNFNGTIHFMWVOQUEJG5ZEMXIRZJB3KKML2LIIWUMJVMRFI6XPHN --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'

[6101187]  rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:05:35
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101187/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101187/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0b775a38a84a4626b2bca4171104df5f

    Command failed on gibba043 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 9820373610 --totp-seed
    U2SEWP7OITKTFWGKDERUSH4SYKVWEOTXJIYB3NZNBIISFDCE6P54BXZ7PODIRQ2I --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'

[6101189]  rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{centos_8}}
-----------------------------------------------------------------
time:   00:06:37
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101189/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101189/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bd24e7b347d24be5903bc8039673a8d0

    Command failed on gibba023 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr
    --format json -n client.0 --cluster ceph user suspend --uid foo'

[6101191]  rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests}
-----------------------------------------------------------------
time:   00:09:38
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101191/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101191/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d55530d1e62c4146b60a4bd6d21b399f

    Command failed on gibba016 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 3326336480 --totp-seed
    MT4EGQOR3KGK7RZFWZ753QOTP6RYZJJW2U6BF3BYQFKICF2Y4RRDHNXDWYLYEJL7 --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'

[6101192]  rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:05:26
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101192/
log:    http://qa-proxy.ceph.com/teuthology/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101192/
sentry: https://sentry.ceph.com/organizations/ceph/?query=e386c0914a18496d988dc9ba1438ad4b

    Command failed on gibba027 with status 17: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create
    --uid foo.client.0 --totp-serial 0511633297 --totp-seed
    QSYJLBWDXHQH3B36PLDSDWB2IWT46CBUC72IBNC6IFRA4636U6MUS2BNRYNSZEYI --totp-
    seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph'



Pass
=================================================================
[6101159] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop}
time:   00:14:09
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101159/

[6101160] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload ubuntu_latest}
time:   00:08:01
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101160/

[6101161] rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{rhel_8} tasks/test_multi}
time:   00:36:27
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101161/

[6101165] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota}
time:   00:11:24
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101165/

[6101166] rgw/tools/{centos_latest cluster tasks}
time:   02:03:57
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101166/

[6101168] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload}
time:   00:09:26
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101168/

[6101176] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota}
time:   00:10:04
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101176/

[6101179] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop}
time:   00:12:51
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101179/

[6101180] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_user_quota ubuntu_latest}
time:   00:06:47
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101180/

[6101183] rgw/tempest/{clusters/fixed-1 frontend/civetweb overrides tasks/rgw_tempest ubuntu_latest}
time:   00:17:50
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101183/

[6101184] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota}
time:   00:09:41
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101184/

[6101186] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload}
time:   00:10:06
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101186/

[6101188] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_bucket_quota ubuntu_latest}
time:   00:07:58
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101188/

[6101190] rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_8} tasks/test_multi}
time:   00:20:47
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101190/

[6101193] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload ubuntu_latest}
time:   00:06:49
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101193/

[6101194] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota}
time:   00:09:55
info:   http://pulpito.ceph.com/ivancich-2021-05-06_14:56:12-rgw-wip-eric-testing-2-distro-basic-gibba/6101194/