2026-02-12T20:29:35.356 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-12T20:29:35.356 INFO:teuthology.results:seed: '6478' 2026-02-12T20:58:05.835 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-12T20:58:05.835 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2026-02-13T08:58:53.036 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2026-02-13T08:58:54.869 INFO:scrape:Found 65 jobs 2026-02-13T08:58:56.386 WARNING:scrape:Missing teuthology log /home/teuthworker/mnt/teuthology/cbodley-2026-02-12_15:40:18-rgw-wip-73090-tentacle-distro-default-trial/46484/teuthology.log 2026-02-13T08:58:57.677 INFO:scrape:Found 6 distinct failure reasons 2026-02-13T08:58:57.677 INFO:scrape:Failure: Command failed (workunit test rgw/s3_user_quota-run.sh) on trial048 with status 22: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=81ef788decab951f81f2991435f179bcbe04d0ee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/s3_user_quota-run.sh' 2026-02-13T08:58:57.677 INFO:scrape:3 jobs: ['46558', '46514', '46464'] 2026-02-13T08:58:57.677 INFO:scrape:suites intersection: ['clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'objectstore/bluestore-bitmap', 'overrides', 'rgw/multifs/{0-install', 'rgw_multipart_upload', 'rgw_ragweed', 'rgw_s3tests', 'rgw_user_quota}', 's3tests-branch', 'tasks/{rgw_bucket_quota', 'ubuntu_latest}'] 2026-02-13T08:58:57.677 INFO:scrape:suites union: ['clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'objectstore/bluestore-bitmap', 'overrides', 'rgw/multifs/{0-install', 'rgw_multipart_upload', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 'rgw_ragweed', 'rgw_s3tests', 'rgw_user_quota}', 's3tests-branch', 'tasks/{rgw_bucket_quota', 'ubuntu_latest}'] 2026-02-13T08:58:57.677 INFO:scrape: 2026-02-13T08:58:57.677 INFO:scrape:Failure: Command failed on trial086 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test_account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|.*test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_services_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*test_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticweb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.*test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds|.*test_object_services.ObjectTest.test_create_object_with_transfer_encoding|.*test_object_expiry.ObjectExpiryTest.test_get_object_after_expiry_time|.*test_object_expiry.ObjectExpiryTest.test_get_object_at_expiry_time|.*test_account_services.AccountTest.test_list_no_account_metadata'" 2026-02-13T08:58:57.677 INFO:scrape:1 jobs: ['46519'] 2026-02-13T08:58:57.677 INFO:scrape:suites: ['clusters/fixed-1', 'frontend/beast', 'ignore-pg-availability', 'overrides', 'rgw/tempest/{0-install', 's3tests-branch', 'tasks/tempest', 'ubuntu_latest}'] 2026-02-13T08:58:57.677 INFO:scrape: 2026-02-13T08:58:57.678 INFO:scrape:Failure: Command failed (s3 tests against rgw) on trial123 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3 and bucket_logging'" 2026-02-13T08:58:57.678 INFO:scrape:2 jobs: ['46449', '46455'] 2026-02-13T08:58:57.678 INFO:scrape:suites intersection: ['ignore-pg-availability', 'overrides', 's3tests-branch'] 2026-02-13T08:58:57.678 INFO:scrape:suites union: ['beast', 'fixed-1', 'ignore-pg-availability', 'overrides', 'rgw/bucket-logging/{0-install', 'rgw/dbstore/{cluster', 's3tests-branch', 'supported-distros/{ubuntu_latest}', 'supported-random-distro$/{centos_latest}', 'tasks/rgw_s3tests}', 'tasks/{s3tests}}'] 2026-02-13T08:58:57.678 INFO:scrape: 2026-02-13T08:58:57.678 INFO:scrape:Failure: No module named 'boto.vendored.six.moves' 2026-02-13T08:58:57.678 INFO:scrape:5 jobs: ['46466', '46516', '46473', '46560', '46536'] 2026-02-13T08:58:57.678 INFO:scrape:suites intersection: ['frontend/beast', 'ignore-pg-availability', 'overrides'] 2026-02-13T08:58:57.678 INFO:scrape:suites union: ['frontend/beast', 'ignore-pg-availability', 'notify', 'objectstore/bluestore-bitmap', 'omap_limits', 'overrides', 'realms/two-zonegroup', 'realms/two-zones', 'rgw/multisite/{clusters', 'rgw/singleton/{all/radosgw-admin', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 'supported-random-distro$/{centos_latest}}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}'] 2026-02-13T08:58:57.678 INFO:scrape: 2026-02-13T08:58:57.678 INFO:scrape:Failure: Command failed (bucket notification tests against different endpoints) on trial092 with status 1: 'BNTESTS_CONF=/home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/bn-tests.client.0.conf /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/virtualenv/bin/python -m nose -s /home/ubuntu/cephtest/ceph/src/test/rgw/bucket_notification/test_bn.py -v -a http_test' 2026-02-13T08:58:57.678 INFO:scrape:4 jobs: ['46501', '46554', '46527', '46468'] 2026-02-13T08:58:57.678 INFO:scrape:suites intersection: ['bluestore-bitmap', 'fixed-2', 'ignore-pg-availability', 'overrides', 'rgw/notifications/{beast', 'valgrind}'] 2026-02-13T08:58:57.678 INFO:scrape:suites union: ['bluestore-bitmap', 'centos_latest', 'fixed-2', 'ignore-pg-availability', 'overrides', 'rgw/notifications/{beast', 'supported-distros/{centos_latest}', 'supported-distros/{ubuntu_latest}', 'tasks/amqp/{0-install', 'tasks/http/{0-install', 'tasks/kafka/{0-install', 'tasks/kafka_failover/{0-install', 'test_amqp}', 'test_http}', 'test_kafka}', 'valgrind}'] 2026-02-13T08:58:57.678 INFO:scrape: 2026-02-13T08:58:57.678 INFO:scrape:Dead: None 2026-02-13T08:58:57.678 INFO:scrape:1 jobs: ['46484'] 2026-02-13T08:58:57.678 INFO:scrape: