2023-08-15T21:14:12.711 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc
2023-08-15T21:14:12.712 INFO:teuthology.results:subset: '111/120000'
2023-08-15T21:14:12.712 INFO:teuthology.results:seed: '2467'
2023-08-15T21:41:19.968 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc
2023-08-15T21:41:19.972 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-08-16T01:05:51.999 INFO:teuthology.results:Tests finished! gathering results...
2023-08-16T01:06:00.017 INFO:scrape:Found 41 jobs
2023-08-16T01:06:00.325 WARNING:scrape:Misunderstood line: Binary file /home/teuthworker/archive/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368809/teuthology.log matches
2023-08-16T01:06:00.326 WARNING:scrape:Misunderstood line: 
2023-08-16T01:06:01.909 INFO:scrape:Found 1 distinct failure reasons
2023-08-16T01:06:01.910 INFO:scrape:Valgrind: 
2023-08-16T01:06:01.910 INFO:scrape:8 jobs: ['7368809', '7368794', '7368790', '7368789', '7368799', '7368775', '7368805', '7368804']
2023-08-16T01:06:01.911 INFO:scrape:suites intersection: []
2023-08-16T01:06:01.911 INFO:scrape:suites union: ['1-ceph-install/install', '2-kms/barbican', '3-rgw/rgw', '4-tests/{s3tests}', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'http', 'ignore-pg-availability', 'inline-data$/{off}', 'mp_reupload', 'msgr-failures/few', 'notify', 'objectstore/bluestore-bitmap', 'omap_limits', 'overrides', 'proto/http', 'ragweed', 'realms/three-zones', 'realms/two-zonegroup', 'reshard', 'rgw/crypt/{0-cluster/fixed-1', 'rgw/dbstore/{cluster', 'rgw/multifs/{clusters/fixed-2', 'rgw/multisite/{clusters', 'rgw/verify/{0-install', 'rgw/website/{clusters/fixed-2', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'tasks/rgw_multipart_upload', 'tasks/rgw_s3tests}', 'tasks/s3tests-website', 'tasks/test_multi}', 'tasks/{cls', 'ubuntu_latest}', 'validater/valgrind}', 'versioning}']
2023-08-16T01:06:01.911 INFO:scrape:
2023-08-16T01:06:01.914 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/
failed:  6
dead:    2
running: 0
waiting: 0
queued:  0
passed:  33


Fail
=================================================================
[7368775]  rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_latest} tasks/test_multi}
-----------------------------------------------------------------
time:   03:39:21
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368775/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368775/
sentry: https://sentry.ceph.com/organizations/ceph/?query=9a754ec1d44d427dbd531b6c3984932c

    rgw multisite test failures

[7368789]  rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:15:13
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368789/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368789/
sentry: https://sentry.ceph.com/organizations/ceph/?query=07dfc96d8ac8473a89d75071c9fe5703

    Command failed on smithi151 with status 1: 'cd
    /home/ubuntu/cephtest/keystone && source /home/ubuntu/cephtest/tox-
    venv/bin/activate && tox -e venv --notest'

[7368790]  rgw/dbstore/{cluster overrides s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/rgw_s3tests}
-----------------------------------------------------------------
time:   01:22:49
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368790/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368790/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1426c8c32bfb4f52ab064b288cd8d55f

    Command failed on smithi088 with status 5: 'adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm
    --uid foo.client.0 --purge-data --cluster ceph'

[7368794]  rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{centos_latest} tasks/test_multi}
-----------------------------------------------------------------
time:   02:40:32
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368794/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368794/
sentry: https://sentry.ceph.com/organizations/ceph/?query=abff238b327b4bca8a4c1f3f3290d9f4

    rgw multisite test failures

[7368804]  rgw/website/{clusters/fixed-2 frontend/beast http overrides s3tests-branch tasks/s3tests-website ubuntu_latest}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368804/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368804/
sentry: https://sentry.ceph.com/organizations/ceph/?query=d8e072e3f6b1454a84cbb774dce58879

    Command failed on smithi039 with status 100: 'sudo apt-get clean'

[7368809]  rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
-----------------------------------------------------------------
time:   00:52:37
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368809/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368809/

    saw valgrind issues



Dead
=================================================================
[7368799]  rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368799/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368799/

    Error reimaging machines: Failed to power on smithi097

[7368805]  rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_multipart_upload ubuntu_latest}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368805/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368805/
sentry: https://sentry.ceph.com/organizations/ceph/?query=a237ffeeda5740c5856ba3f4b6bb6ce0

    SSH connection to smithi039 was lost: 'sudo DEBIAN_FRONTEND=noninteractive
    apt-get -y install linux-image-generic'



Pass
=================================================================
[7368774] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop}
time:   00:20:03
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368774/

[7368776] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests}
time:   00:24:20
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368776/

[7368777] rgw/upgrade/{1-install/quincy/{distro$/{centos_latest} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast objectstore/bluestore-bitmap overrides}
time:   00:21:58
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368777/

[7368778] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:39:46
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368778/

[7368779] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:12:55
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368779/

[7368780] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_20.04}}
time:   00:15:49
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368780/

[7368781] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_20.04}}
time:   00:13:46
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368781/

[7368782] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:21:38
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368782/

[7368783] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota}
time:   00:16:51
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368783/

[7368784] rgw/upgrade/{1-install/pacific/{distro$/{centos_8.stream} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast objectstore/bluestore-bitmap overrides}
time:   00:22:53
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368784/

[7368785] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_20.04} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}
time:   01:07:07
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368785/

[7368786] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{centos_latest}}
time:   00:16:28
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368786/

[7368787] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_user_quota ubuntu_latest}
time:   00:13:41
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368787/

[7368788] rgw/cloud-transition/{cluster overrides s3tests-branch supported-random-distro$/{ubuntu_20.04} tasks/cloud_transition_s3tests}
time:   00:31:47
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368788/

[7368791] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop}
time:   00:21:14
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368791/

[7368792] rgw/lifecycle/{cluster overrides s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/rgw_s3tests}
time:   00:28:07
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368792/

[7368793] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_bucket_quota ubuntu_latest}
time:   00:15:16
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368793/

[7368795] rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}}
time:   00:39:02
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368795/

[7368796] rgw/service-token/{clusters/fixed-1 frontend/beast overrides tasks/service-token ubuntu_latest}
time:   00:17:22
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368796/

[7368797] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_latest}}
time:   00:16:44
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368797/

[7368798] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast s3tests-branch supported-random-distro$/{rhel_8} tasks/{0-install first ststests}}
time:   00:33:30
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368798/

[7368800] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota}
time:   00:17:15
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368800/

[7368801] rgw/tools/{centos_latest cluster tasks}
time:   02:45:21
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368801/

[7368802] rgw/upgrade/{1-install/pacific/{distro$/{ubuntu_20.04} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast objectstore/bluestore-bitmap overrides}
time:   00:15:03
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368802/

[7368803] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/lockdep}
time:   00:44:08
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368803/

[7368806] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_20.04}}
time:   00:16:27
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368806/

[7368807] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload}
time:   00:17:45
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368807/

[7368808] rgw/upgrade/{1-install/quincy/{distro$/{ubuntu_20.04} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast objectstore/bluestore-bitmap overrides}
time:   00:15:15
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368808/

[7368810] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:13:19
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368810/

[7368811] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_latest}}
time:   00:20:15
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368811/

[7368812] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}}
time:   00:14:03
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368812/

[7368813] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:21:39
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368813/

[7368814] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{centos_latest}}
time:   00:18:44
info:   http://pulpito.ceph.com/yuriw-2023-08-15_16:15:59-rgw-wip-yuri-testing-2023-08-14-0749-reef-distro-default-smithi/7368814/