2023-05-31T19:26:08.031 INFO:root:teuthology version: 0.0.1.dev111+gfa97ddd
2023-05-31T19:26:08.032 INFO:teuthology.results:subset: '111/12000'
2023-05-31T19:26:08.032 INFO:teuthology.results:seed: '2752'
2023-05-31T19:51:05.608 INFO:root:teuthology version: 0.0.1.dev111+gfa97ddd
2023-05-31T19:51:05.612 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-05-31T22:34:18.263 INFO:teuthology.results:Tests finished! gathering results...
2023-05-31T22:34:21.330 INFO:scrape:Found 41 jobs
2023-05-31T22:34:21.660 WARNING:scrape:Misunderstood line: Binary file /home/teuthworker/archive/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292319/teuthology.log matches
2023-05-31T22:34:21.661 WARNING:scrape:Misunderstood line: 
2023-05-31T22:34:36.637 INFO:scrape:Found 1 distinct failure reasons
2023-05-31T22:34:36.638 INFO:scrape:Valgrind: 
2023-05-31T22:34:36.638 INFO:scrape:6 jobs: ['7292319', '7292304', '7292309', '7292285', '7292316', '7292305']
2023-05-31T22:34:36.639 INFO:scrape:suites intersection: []
2023-05-31T22:34:36.639 INFO:scrape:suites union: ['1-ceph-install/install', '2-kms/kmip', '3-rgw/rgw', '4-tests/{s3tests}', 'bluestore-bitmap', 'clusters/fixed-2', 'datacache/rgw-datacache', 'fixed-2', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'msgr-failures/few', 'notify', 'objectstore/bluestore-bitmap', 'omap_limits', 'overrides', 'proto/http', 'ragweed', 'realms/three-zones', 'realms/two-zonegroup', 'reshard', 'rgw/crypt/{0-cluster/fixed-1', 'rgw/multisite/{clusters', 'rgw/notifications/{beast', 'rgw/tempest/{clusters/fixed-1', 'rgw/verify/{0-install', 'rgw_pool_type/ec-profile', 's3tests-branch', 's3tests-java', 's3tests}', 'sharding$/{default}', 'striping$/{stripe-greater-than-chunk}', 'supported-all-distro$/{centos_8}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'tasks/rgw_tempest', 'tasks/test_multi}', 'tasks/{0-install', 'tasks/{cls', 'test_amqp', 'test_kafka', 'test_others}}', 'ubuntu_latest}', 'validater/valgrind}']
2023-05-31T22:34:36.640 INFO:scrape:
2023-05-31T22:34:36.642 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/
failed:  6
dead:    0
running: 0
waiting: 0
queued:  0
passed:  35


Fail
=================================================================
[7292285]  rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{centos_8} tasks/test_multi}
-----------------------------------------------------------------
time:   02:03:21
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292285/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292285/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8d201981081f4c40857626305f95bc8e

    rgw multisite test failures

[7292304]  rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{ubuntu_latest} tasks/test_multi}
-----------------------------------------------------------------
time:   02:22:20
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292304/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292304/
sentry: https://sentry.ceph.com/organizations/ceph/?query=2cbaa2481dde4ca799d96099fc689d28

    rgw multisite test failures

[7292305]  rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}}
-----------------------------------------------------------------
time:   01:38:42
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292305/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292305/
sentry: https://sentry.ceph.com/organizations/ceph/?query=35db1952517e441dbc0bc7d75d82a173

    Command failed (bucket notification tests against different endpoints) on
    smithi084 with status 1: 'BNTESTS_CONF=/home/ubuntu/cephtest/ceph/src/test/
    rgw/bucket_notification/bn-tests.client.0.conf /home/ubuntu/cephtest/ceph/s
    rc/test/rgw/bucket_notification/virtualenv/bin/python -m nose -s
    /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py -v
    -a kafka_test'

[7292309]  rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest}
-----------------------------------------------------------------
time:   00:15:08
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292309/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292309/
sentry: https://sentry.ceph.com/organizations/ceph/?query=1ae87a84217b4ccd9556f58593b87a31

    Command failed on smithi146 with status 1: 'cd
    /home/ubuntu/cephtest/keystone && source /home/ubuntu/cephtest/tox-
    venv/bin/activate && tox -e venv --notest'

[7292316]  rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_latest}}
-----------------------------------------------------------------
time:   00:10:53
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292316/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292316/
sentry: https://sentry.ceph.com/organizations/ceph/?query=80988b8389404f9299b709d6f2215103

    Command failed on smithi035 with status 100:
    'DEBIAN_FRONTEND=noninteractive sudo -E apt-get -y --force-yes install
    python-dev'

[7292319]  rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
-----------------------------------------------------------------
time:   01:15:17
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292319/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292319/

    saw valgrind issues



Pass
=================================================================
[7292284] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop}
time:   00:17:09
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292284/

[7292286] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests}
time:   00:23:09
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292286/

[7292287] rgw/upgrade/{1-install/quincy/{distro$/{centos_latest} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast objectstore/bluestore-bitmap overrides}
time:   00:18:52
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292287/

[7292288] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{ubuntu_latest} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:42:47
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292288/

[7292289] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:11:56
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292289/

[7292290] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_20.04}}
time:   00:15:01
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292290/

[7292291] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_8}}
time:   00:15:59
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292291/

[7292292] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:20:15
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292292/

[7292293] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota}
time:   00:15:59
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292293/

[7292294] rgw/upgrade/{1-install/pacific/{distro$/{centos_latest} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast objectstore/bluestore-bitmap overrides}
time:   00:23:56
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292294/

[7292295] rgw/verify/{0-install clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{centos_8} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   00:49:02
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292295/

[7292296] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{centos_8}}
time:   00:16:14
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292296/

[7292297] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_user_quota ubuntu_latest}
time:   00:12:32
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292297/

[7292298] rgw/cloud-transition/{cluster overrides s3tests-branch supported-random-distro$/{centos_8} tasks/cloud_transition_s3tests}
time:   00:33:14
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292298/

[7292299] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{rhel_8}}
time:   00:26:09
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292299/

[7292300] rgw/dbstore/{cluster overrides s3tests-branch supported-random-distro$/{rhel_8} tasks/rgw_s3tests}
time:   00:20:51
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292300/

[7292301] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop}
time:   00:17:27
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292301/

[7292302] rgw/lifecycle/{cluster overrides s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/rgw_s3tests}
time:   00:26:45
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292302/

[7292303] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_bucket_quota ubuntu_latest}
time:   00:13:49
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292303/

[7292306] rgw/service-token/{clusters/fixed-1 frontend/beast overrides tasks/service-token ubuntu_latest}
time:   00:16:34
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292306/

[7292307] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}}
time:   00:22:57
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292307/

[7292308] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast s3tests-branch supported-random-distro$/{centos_8} tasks/{0-install first ststests}}
time:   00:30:19
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292308/

[7292310] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota}
time:   00:17:24
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292310/

[7292311] rgw/tools/{centos_latest cluster tasks}
time:   02:46:28
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292311/

[7292312] rgw/upgrade/{1-install/pacific/{distro$/{ubuntu_20.04} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast objectstore/bluestore-bitmap overrides}
time:   00:13:35
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292312/

[7292313] rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/replicated s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{rhel_8} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:35:46
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292313/

[7292314] rgw/website/{clusters/fixed-2 frontend/beast http overrides s3tests-branch tasks/s3tests-website ubuntu_latest}
time:   00:22:35
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292314/

[7292315] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_multipart_upload ubuntu_latest}
time:   00:12:57
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292315/

[7292317] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload}
time:   00:15:45
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292317/

[7292318] rgw/upgrade/{1-install/quincy/{distro$/{ubuntu_20.04} install overrides} 2-setup 3-upgrade-sequence/rgws-then-osds cluster frontend/beast objectstore/bluestore-bitmap overrides}
time:   00:13:41
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292318/

[7292320] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:11:56
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292320/

[7292321] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_latest}}
time:   00:18:54
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292321/

[7292322] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_8}}
time:   00:14:01
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292322/

[7292323] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:20:02
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292323/

[7292324] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{centos_8}}
time:   00:22:45
info:   http://pulpito.ceph.com/yuriw-2023-05-31_19:25:20-rgw-reef-release-distro-default-smithi/7292324/