2023-11-28T14:22:28.128 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5
2023-11-28T14:22:28.128 INFO:teuthology.results:seed: '280'
2023-11-28T15:19:42.980 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5
2023-11-28T15:19:42.983 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2023-11-28T19:21:03.019 INFO:teuthology.results:Tests finished! gathering results...
2023-11-28T19:21:09.801 INFO:scrape:Found 72 jobs
2023-11-28T19:21:09.802 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/cbodley-2023-11-27_20:42:48-rgw-wip-cbodley-testing-distro-default-smithi/7468513/teuthology.log
2023-11-28T19:21:10.456 INFO:scrape:Found 5 distinct failure reasons
2023-11-28T19:21:10.457 INFO:scrape:Dead: None
2023-11-28T19:21:10.457 INFO:scrape:1 jobs: ['7468513']
2023-11-28T19:21:10.457 INFO:scrape:
2023-11-28T19:21:10.458 INFO:scrape:Failure: ['<https://docs.pagure.org/copr.copr/user_documentation.html#what-i-can-build-in-copr>,', 'Bugzilla. In case of problems, contact the owner of this repository.', 'Enabling a Copr repository. Please note that this repository is not part', 'Error: Failed to connect to https://copr.fedorainfracloud.org/coprs/ceph/python3-asyncssh/repo/epel-8/dnf.repo?arch=x86_64: Network is unreachable', 'Please do not file bug reports about these packages in Fedora', 'The Fedora Project does not exercise any power over the contents of', 'and packages are not held to any quality or security level.', 'of the main distribution, and quality may vary.', 'this repository beyond the rules outlined in the Copr FAQ at']
2023-11-28T19:21:10.458 INFO:scrape:1 jobs: ['7468514']
2023-11-28T19:21:10.458 INFO:scrape:suites: ['2-setup', '3-upgrade-sequence/osds-then-rgws', 'cluster', 'frontend/beast', 'ignore-pg-availability', 'install', 'objectstore/bluestore-bitmap', 'overrides}', 'rgw/upgrade/{1-install/pacific/{distro$/{centos_8.stream}']
2023-11-28T19:21:10.458 INFO:scrape:
2023-11-28T19:21:10.458 INFO:scrape:Failure: Command failed on smithi114 with status 100: 'DEBIAN_FRONTEND=noninteractive sudo -E apt-get -y --force-yes install python-dev'
2023-11-28T19:21:10.458 INFO:scrape:1 jobs: ['7468522']
2023-11-28T19:21:10.458 INFO:scrape:suites: ['1-ceph-install/install', '2-kms/kmip', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability', 'rgw/crypt/{0-cluster/fixed-1', 's3tests-branch', 'ubuntu_latest}']
2023-11-28T19:21:10.458 INFO:scrape:
2023-11-28T19:21:10.459 INFO:scrape:Failure: rgw multisite test failures
2023-11-28T19:21:10.459 INFO:scrape:2 jobs: ['7468506', '7468542']
2023-11-28T19:21:10.459 INFO:scrape:suites intersection: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'rgw/multisite/{clusters', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}']
2023-11-28T19:21:10.459 INFO:scrape:suites union: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zones', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}']
2023-11-28T19:21:10.459 INFO:scrape:
2023-11-28T19:21:10.459 INFO:scrape:Failure: Command failed (workunit test rgw/run-bucket-check.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0948a11d490220e02b84c8df6829aedaa1847bec TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh'
2023-11-28T19:21:10.460 INFO:scrape:1 jobs: ['7468525']
2023-11-28T19:21:10.460 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{single}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/valgrind}', 'versioning}']
2023-11-28T19:21:10.460 INFO:scrape: