2023-06-10T05:40:55.804 INFO:root:teuthology version: 0.0.1.dev120+g2ef0dcd
2023-06-10T05:40:55.805 INFO:teuthology.results:subset: '111/12000'
2023-06-10T05:40:55.805 INFO:teuthology.results:seed: '9157'
2023-06-10T06:15:26.849 INFO:root:teuthology version: 0.0.1.dev120+g2ef0dcd
2023-06-10T06:15:26.853 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-06-10T14:58:05.462 INFO:teuthology.results:Tests finished! gathering results...
2023-06-10T14:58:08.525 INFO:scrape:Found 41 jobs
2023-06-10T14:58:08.806 WARNING:scrape:Misunderstood line: Binary file /home/teuthworker/archive/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300702/teuthology.log matches
2023-06-10T14:58:08.807 WARNING:scrape:Misunderstood line: 
2023-06-10T14:58:10.822 INFO:scrape:Found 1 distinct failure reasons
2023-06-10T14:58:10.823 INFO:scrape:Valgrind: 
2023-06-10T14:58:10.824 INFO:scrape:5 jobs: ['7300702', '7300668', '7300699', '7300687', '7300692']
2023-06-10T14:58:10.824 INFO:scrape:suites intersection: []
2023-06-10T14:58:10.824 INFO:scrape:suites union: ['1-ceph-install/install', '2-kms/kmip', '3-rgw/rgw', '4-tests/{s3tests}', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'msgr-failures/few', 'notify', 'objectstore/bluestore-bitmap', 'omap_limits', 'overrides', 'proto/http', 'ragweed', 'realms/three-zones', 'realms/two-zonegroup', 'reshard', 'rgw/crypt/{0-cluster/fixed-1', 'rgw/multisite/{clusters', 'rgw/tempest/{clusters/fixed-1', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests-branch', 's3tests-java', 's3tests}', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_20.04}}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/rgw_tempest', 'tasks/test_multi}', 'tasks/{cls', 'ubuntu_latest}', 'validater/valgrind}']
2023-06-10T14:58:10.825 INFO:scrape:
2023-06-10T14:58:10.827 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/
failed:  5
dead:    0
running: 0
waiting: 0
queued:  0
passed:  36


Fail
=================================================================
[7300668]  rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_8} tasks/test_multi}
-----------------------------------------------------------------
time:   08:18:11
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300668/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300668/
sentry: https://sentry.ceph.com/organizations/ceph/?query=719af280c6434170aaf7d43d4d5776ac

    rgw multisite test failures

[7300687]  rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{rhel_8} tasks/test_multi}
-----------------------------------------------------------------
time:   03:54:06
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300687/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300687/
sentry: https://sentry.ceph.com/organizations/ceph/?query=aa86c1bf5afa4d2ca24da5a893f3ac21

    rgw multisite test failures

[7300692]  rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest}
-----------------------------------------------------------------
time:   00:15:25
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300692/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300692/
sentry: https://sentry.ceph.com/organizations/ceph/?query=6df7eba3ee1b4d9599a34709f3721505

    Command failed on smithi150 with status 1: 'cd
    /home/ubuntu/cephtest/keystone && source /home/ubuntu/cephtest/tox-
    venv/bin/activate && tox -e venv --notest'

[7300699]  rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_20.04}}
-----------------------------------------------------------------
time:   00:18:06
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300699/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300699/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2b4e75ddc6f24b0eb33ece8547baef83

    Command failed (s3 tests against rgw) on smithi084 with status 1: "source
    /home/ubuntu/cephtest/tox-venv/bin/activate && cd
    /home/ubuntu/cephtest/s3-tests-client.0 &&
    S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf
    BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg
    REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not
    fails_on_rgw and not lifecycle_expiration and not test_of_sts and not
    webidentity_test and not fails_with_subdomain and not sse_s3'"

[7300702]  rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   01:13:14
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300702/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300702/

    saw valgrind issues



Pass
=================================================================
[7300667] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop}
time:   00:16:49
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300667/

[7300669] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests}
time:   00:22:41
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300669/

[7300670] rgw/upgrade/{1-install/quincy/{distro$/{ubuntu_20.04} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast objectstore/bluestore-bitmap overrides}
time:   00:13:42
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300670/

[7300671] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:42:16
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300671/

[7300672] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:12:03
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300672/

[7300673] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_latest}}
time:   00:19:12
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300673/

[7300674] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_20.04}}
time:   00:11:47
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300674/

[7300675] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:20:10
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300675/

[7300676] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota}
time:   00:15:53
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300676/

[7300677] rgw/upgrade/{1-install/pacific/{distro$/{ubuntu_20.04} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast objectstore/bluestore-bitmap overrides}
time:   00:13:30
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300677/

[7300678] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_8} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   01:19:30
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300678/

[7300679] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{centos_8}}
time:   00:36:35
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300679/

[7300680] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_user_quota ubuntu_latest}
time:   00:12:39
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300680/

[7300681] rgw/cloud-transition/{cluster overrides s3tests-branch supported-random-distro$/{rhel_8} tasks/cloud_transition_s3tests}
time:   00:34:49
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300681/

[7300682] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{centos_8}}
time:   00:54:52
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300682/

[7300683] rgw/dbstore/{cluster overrides s3tests-branch supported-random-distro$/{centos_8} tasks/rgw_s3tests}
time:   00:42:50
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300683/

[7300684] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop}
time:   00:17:17
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300684/

[7300685] rgw/lifecycle/{cluster overrides s3tests-branch supported-random-distro$/{rhel_8} tasks/rgw_s3tests}
time:   00:31:20
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300685/

[7300686] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_bucket_quota ubuntu_latest}
time:   00:13:38
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300686/

[7300688] rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}}
time:   00:43:56
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300688/

[7300689] rgw/service-token/{clusters/fixed-1 frontend/beast overrides tasks/service-token ubuntu_latest}
time:   00:17:12
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300689/

[7300690] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_20.04}}
time:   00:12:05
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300690/

[7300691] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install first ststests}}
time:   00:31:13
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300691/

[7300693] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota}
time:   00:16:09
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300693/

[7300694] rgw/tools/{centos_latest cluster tasks}
time:   03:03:33
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300694/

[7300695] rgw/upgrade/{1-install/pacific/{distro$/{ubuntu_20.04} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast objectstore/bluestore-bitmap overrides}
time:   00:13:48
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300695/

[7300696] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_20.04} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:34:00
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300696/

[7300697] rgw/website/{clusters/fixed-2 frontend/beast http overrides s3tests-branch tasks/s3tests-website ubuntu_latest}
time:   00:23:19
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300697/

[7300698] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_multipart_upload ubuntu_latest}
time:   00:12:59
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300698/

[7300700] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload}
time:   00:16:42
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300700/

[7300701] rgw/upgrade/{1-install/quincy/{distro$/{centos_latest} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast objectstore/bluestore-bitmap overrides}
time:   00:28:54
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300701/

[7300703] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:12:15
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300703/

[7300704] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{centos_8}}
time:   00:16:38
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300704/

[7300705] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_20.04}}
time:   00:12:01
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300705/

[7300706] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:20:20
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300706/

[7300707] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{rhel_8}}
time:   00:21:23
info:   http://pulpito.ceph.com/yuriw-2023-06-09_21:04:16-rgw-reef-distro-default-smithi/7300707/