2021-02-22T19:47:13.750 INFO:root:teuthology version: 1.1.0-9a6b849c 2021-02-22T19:47:13.753 INFO:teuthology.results:subset: '1/2' 2021-02-22T19:47:13.753 INFO:teuthology.results:seed: '5552' 2021-02-22T20:33:48.618 INFO:root:teuthology version: 1.1.0-9a6b849c 2021-02-22T20:33:48.641 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-02-22T21:30:49.902 INFO:teuthology.results:Tests finished! gathering results... 2021-02-22T21:33:01.744 INFO:scrape:Found 74 jobs 2021-02-22T21:34:54.111 INFO:scrape:Found 9 distinct failure reasons 2021-02-22T21:34:54.112 INFO:scrape:Failure: Command failed on smithi105 with status 22: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid fud --bucket myfoo --bucket-id 1aa3d103-90cc-469f-a5ec-306b603fc9bb.4217.1' 2021-02-22T21:34:54.113 INFO:scrape:6 jobs: ['5904132', '5904121', '5904096', '5904084', '5904071', '5904108'] 2021-02-22T21:34:54.113 INFO:scrape:suites intersection: ['frontend/civetweb', 'overrides', 'rgw/singleton/{all/radosgw-admin'] 2021-02-22T21:34:54.114 INFO:scrape:suites union: ['frontend/civetweb', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'rgw/singleton/{all/radosgw-admin', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{rhel_8}}', 'supported-random-distro$/{ubuntu_latest}}'] 2021-02-22T21:34:54.114 INFO:scrape: 2021-02-22T21:34:54.115 INFO:scrape:Failure: Command failed on smithi036 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 75f280bba0c691775574cbebd70c03a2d1df59d1' 2021-02-22T21:34:54.115 INFO:scrape:4 jobs: ['5904138', '5904140', '5904137', '5904130'] 2021-02-22T21:34:54.116 INFO:scrape:suites intersection: ['cls', 'clusters/fixed-2', 'msgr-failures/few', 'overrides', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 's3tests-java', 's3tests}', 'striping$/{stripe-equals-chunk}', 'tasks/{0-install'] 2021-02-22T21:34:54.116 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'frontend/beast', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{0-install', 'validater/lockdep}', 'validater/valgrind}'] 2021-02-22T21:34:54.117 INFO:scrape: 2021-02-22T21:34:54.117 INFO:scrape:Failure: machine smithi134.front.sepia.ceph.com is locked by scheduled_jdillaman@cube-1, not scheduled_dang@sidious.arb.redhat.com 2021-02-22T21:34:54.117 INFO:scrape:1 jobs: ['5904134'] 2021-02-22T21:34:54.118 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'tasks/{0-install', 'validater/lockdep}'] 2021-02-22T21:34:54.118 INFO:scrape: 2021-02-22T21:34:54.119 INFO:scrape:Failure: "2021-02-22T20:30:04.498401+0000 mon.a (mon.0) 232 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log 2021-02-22T21:34:54.119 INFO:scrape:1 jobs: ['5904109'] 2021-02-22T21:34:54.120 INFO:scrape:suites: ['clusters/fixed-2', 'frontend/civetweb', 'objectstore/filestore-xfs', 'overrides', 'rgw/sts/{centos_latest', 'rgw_pool_type/ec-profile', 'ststests', 'tasks/{0-install', 'webidentity}}'] 2021-02-22T21:34:54.120 INFO:scrape: 2021-02-22T21:34:54.120 INFO:scrape:Failure: rgw multisite test failures 2021-02-22T21:34:54.121 INFO:scrape:1 jobs: ['5904107'] 2021-02-22T21:34:54.122 INFO:scrape:suites: ['frontend/civetweb', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'rgw/multisite/{clusters', 'tasks/test_multi', 'valgrind}'] 2021-02-22T21:34:54.122 INFO:scrape: 2021-02-22T21:34:54.123 INFO:scrape:Failure: SSH connection to smithi134 was lost: 'sudo apt-get clean' 2021-02-22T21:34:54.123 INFO:scrape:1 jobs: ['5904117'] 2021-02-22T21:34:54.124 INFO:scrape:suites: ['frontend/civetweb', 'objectstore/bluestore-bitmap', 'overrides', 'rgw/multifs/{clusters/fixed-2', 'rgw_pool_type/ec-profile', 'tasks/rgw_user_quota}'] 2021-02-22T21:34:54.124 INFO:scrape: 2021-02-22T21:34:54.125 INFO:scrape:Failure: Command failed on smithi037 with status 1: "cd /home/ubuntu/cephtest/tempest && source .tox/venv/bin/activate && tempest run --workspace-path /home/ubuntu/cephtest/tempest/workspace.yaml --workspace rgw --regex '^tempest.api.object_storage' --black-regex '.*test_account_quotas_negative.AccountQuotasNegativeTest.test_user_modify_quota|.*test_container_acl_negative.ObjectACLsNegativeTest.*|.*test_container_services_negative.ContainerNegativeTest.test_create_container_metadata_.*|.*test_container_staticweb.StaticWebTest.test_web_index|.*test_container_staticweb.StaticWebTest.test_web_listing_css|.*test_container_synchronization.*|.*test_object_services.PublicObjectTest.test_access_public_container_object_without_using_creds'" 2021-02-22T21:34:54.125 INFO:scrape:2 jobs: ['5904110', '5904073'] 2021-02-22T21:34:54.126 INFO:scrape:suites intersection: ['overrides', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}'] 2021-02-22T21:34:54.126 INFO:scrape:suites union: ['frontend/beast', 'frontend/civetweb', 'overrides', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}'] 2021-02-22T21:34:54.127 INFO:scrape: 2021-02-22T21:34:54.127 INFO:scrape:Failure: Command failed (workunit test rgw/test_rgw_orphan_list.sh) on smithi160 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=75f280bba0c691775574cbebd70c03a2d1df59d1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_orphan_list.sh' 2021-02-22T21:34:54.128 INFO:scrape:1 jobs: ['5904075'] 2021-02-22T21:34:54.128 INFO:scrape:suites: ['cluster', 'rgw/tools/{centos_latest', 'tasks}'] 2021-02-22T21:34:54.128 INFO:scrape: 2021-02-22T21:34:54.129 INFO:scrape:Failure: Could not reconnect to ubuntu@smithi134.front.sepia.ceph.com 2021-02-22T21:34:54.129 INFO:scrape:1 jobs: ['5904116'] 2021-02-22T21:34:54.130 INFO:scrape:suites: ['cls', 'clusters/fixed-2', 'frontend/civetweb', 'msgr-failures/few', 'objectstore/filestore-xfs', 'overrides', 'proto/http', 'ragweed', 'reshard', 'rgw/verify/{centos_latest', 'rgw_pool_type/replicated', 's3tests-java', 's3tests}', 'sharding$/{default}', 'striping$/{stripe-equals-chunk}', 'tasks/{0-install', 'validater/lockdep}'] 2021-02-22T21:34:54.130 INFO:scrape: