2022-10-06T19:42:19.043 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9
2022-10-06T19:42:19.044 INFO:teuthology.results:subset: '111/120000'
2022-10-06T19:42:19.044 INFO:teuthology.results:seed: '6665'
2022-10-06T20:27:53.581 INFO:root:teuthology version: 1.1.1.dev427+g35ea38a9
2022-10-06T20:27:53.601 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2022-10-07T08:07:47.906 INFO:teuthology.results:Tests finished! gathering results...
2022-10-07T08:07:51.561 INFO:scrape:Found 42 jobs
2022-10-07T08:07:53.314 INFO:scrape:Found 3 distinct failure reasons
2022-10-07T08:07:53.315 INFO:scrape:Failure: rgw multisite test failures
2022-10-07T08:07:53.315 INFO:scrape:2 jobs: ['7056640', '7056618']
2022-10-07T08:07:53.316 INFO:scrape:suites intersection: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'rgw/multisite/{clusters', 'tasks/test_multi}']
2022-10-07T08:07:53.316 INFO:scrape:suites union: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zones', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}']
2022-10-07T08:07:53.317 INFO:scrape:
2022-10-07T08:07:53.317 INFO:scrape:Failure: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=9cfd351ab2cf5da8059a5166ce2646107a06f800
2022-10-07T08:07:53.318 INFO:scrape:1 jobs: ['7056615']
2022-10-07T08:07:53.318 INFO:scrape:suites: ['hadoop/default', 'overrides', 'rgw/hadoop-s3a/{clusters/fixed-2', 's3a-hadoop}']
2022-10-07T08:07:53.318 INFO:scrape:
2022-10-07T08:07:53.319 INFO:scrape:Dead: b'2022-10-07T07:56:16.614 DEBUG:teuthology.exit:Finished running handlers'
2022-10-07T08:07:53.319 INFO:scrape:1 jobs: ['7056619']
2022-10-07T08:07:53.320 INFO:scrape:suites: ['bluestore-bitmap', 'fixed-2', 'overrides', 'rgw/notifications/{beast', 'supported-all-distro$/{centos_8}', 'tasks/{0-install', 'test_amqp', 'test_kafka', 'test_others}}']
2022-10-07T08:07:53.320 INFO:scrape:
2022-10-07T08:07:53.322 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi
=================================================================
info:    http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/
failed:  3
dead:    1
running: 0
waiting: 0
queued:  0
passed:  38


Fail
=================================================================
[7056615]  rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop}
-----------------------------------------------------------------
time:   00:04:44
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056615/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056615/
sentry: https://sentry.ceph.com/organizations/ceph/?query=bb69890d38d74cd88c378b3e43370bc0

    Failed to fetch package version from https://shaman.ceph.com/api/search/?st
    atus=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1
    =9cfd351ab2cf5da8059a5166ce2646107a06f800

[7056618]  rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zones supported-random-distro$/{rhel_8} tasks/test_multi}
-----------------------------------------------------------------
time:   02:05:15
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056618/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056618/
sentry: https://sentry.ceph.com/organizations/ceph/?query=8dd447fa60af4565917ff8afc3f5a6da

    rgw multisite test failures

[7056640]  rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{ubuntu_latest} tasks/test_multi}
-----------------------------------------------------------------
time:   01:35:16
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056640/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056640/
sentry: https://sentry.ceph.com/organizations/ceph/?query=0e1391c3dcef4c2eb655d6f2d0d30f4e

    rgw multisite test failures



Dead
=================================================================
[7056619]  rgw/notifications/{beast bluestore-bitmap fixed-2 overrides supported-all-distro$/{centos_8} tasks/{0-install test_amqp test_kafka test_others}}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056619/
log:    http://qa-proxy.ceph.com/teuthology/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056619/

    hit max job timeout



Pass
=================================================================
[7056612] rgw/cloud-transition/{cluster overrides s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/cloud_transition_s3tests}
time:   00:26:48
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056612/

[7056613] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{centos_8}}
time:   00:17:54
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056613/

[7056614] rgw/dbstore/{cluster overrides s3tests-branch supported-random-distro$/{rhel_8} tasks/rgw_s3tests}
time:   00:20:20
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056614/

[7056616] rgw/lifecycle/{cluster overrides s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/rgw_s3tests}
time:   00:20:56
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056616/

[7056617] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_multipart_upload ubuntu_latest}
time:   00:07:25
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056617/

[7056620] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_latest}}
time:   00:08:51
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056620/

[7056621] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast s3tests-branch supported-random-distro$/{ubuntu_latest} tasks/{0-install first ststests}}
time:   00:24:55
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056621/

[7056622] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest}
time:   00:19:01
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056622/

[7056623] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota}
time:   00:11:13
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056623/

[7056624] rgw/tools/{centos_latest cluster tasks}
time:   02:43:51
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056624/

[7056625] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:28:08
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056625/

[7056626] rgw/website/{clusters/fixed-2 frontend/beast http overrides s3tests-branch tasks/s3tests-website ubuntu_latest}
time:   00:15:16
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056626/

[7056627] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload}
time:   00:10:42
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056627/

[7056628] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_latest}}
time:   00:11:30
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056628/

[7056629] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec s3tests-branch tasks/rgw_ragweed ubuntu_latest}
time:   00:06:32
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056629/

[7056630] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{rhel_8}}
time:   00:15:00
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056630/

[7056631] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests}
time:   00:16:26
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056631/

[7056632] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   00:58:04
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056632/

[7056633] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{rhel_8}}
time:   00:16:51
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056633/

[7056634] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_s3tests ubuntu_latest}
time:   00:11:49
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056634/

[7056635] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{rhel_8}}
time:   00:15:16
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056635/

[7056636] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota}
time:   00:10:05
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056636/

[7056637] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_latest}}
time:   00:11:02
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056637/

[7056638] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop}
time:   00:16:50
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056638/

[7056639] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile s3tests-branch tasks/rgw_user_quota ubuntu_latest}
time:   00:06:44
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056639/

[7056641] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}}
time:   00:15:18
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056641/

[7056642] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_bucket_quota}
time:   00:11:20
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056642/

[7056643] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/lockdep}
time:   00:25:51
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056643/

[7056644] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_multipart_upload}
time:   00:10:28
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056644/

[7056645] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{ubuntu_latest}}
time:   00:11:08
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056645/

[7056646] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec s3tests-branch tasks/rgw_bucket_quota ubuntu_latest}
time:   00:08:14
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056646/

[7056647] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_8}}
time:   00:10:16
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056647/

[7056648] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/filestore-xfs s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_s3tests}
time:   00:17:12
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056648/

[7056649] rgw/verify/{0-install centos_latest clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} tasks/{cls ragweed reshard s3tests-java s3tests} validater/valgrind}
time:   00:52:58
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056649/

[7056650] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability s3tests-branch supported-random-distro$/{rhel_8}}
time:   00:17:19
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056650/

[7056651] rgw/multifs/{clusters/fixed-2 frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated s3tests-branch tasks/rgw_multipart_upload ubuntu_latest}
time:   00:06:58
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056651/

[7056652] rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{rhel_8}}
time:   00:15:03
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056652/

[7056653] rgw/thrash/{clusters/fixed-2 frontend/beast install objectstore/bluestore-bitmap s3tests-branch thrasher/default thrashosds-health ubuntu_latest workload/rgw_user_quota}
time:   00:11:33
info:   http://pulpito.ceph.com/yuriw-2022-10-05_21:22:08-rgw-main-distro-default-smithi/7056653/