2024-02-16T10:06:56.288 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-02-16T10:06:56.288 INFO:teuthology.results:seed: '0' 2024-02-16T10:50:00.437 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-02-16T10:50:00.466 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-02-16T22:21:20.825 INFO:teuthology.results:Tests finished! gathering results... 2024-02-16T22:21:26.097 INFO:scrape:Found 71 jobs 2024-02-16T22:21:29.559 WARNING:scrape:Not-understood crash indication 2024-02-16T12:00:33.549 INFO:tasks.rgw.c2.client.0.smithi165.stderr:daemon-helper: command crashed with signal 11 2024-02-16T22:21:29.559 WARNING:scrape:Not-understood crash indication 2024-02-16T13:00:47.736 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi165.stderr: INFO: daemon-helper: command crashed with signal 11 2024-02-16T22:21:29.560 WARNING:scrape:Not-understood crash indication 2024-02-16T22:21:30.152 WARNING:scrape:Not-understood crash indication 2024-02-16T12:00:33.549 INFO:tasks.rgw.c2.client.0.smithi165.stderr:daemon-helper: command crashed with signal 11 2024-02-16T22:21:30.152 WARNING:scrape:Not-understood crash indication 2024-02-16T13:00:47.736 INFO:tasks.rgw_multisite_tests:tasks.rgw.c2.client.0.smithi165.stderr: INFO: daemon-helper: command crashed with signal 11 2024-02-16T22:21:30.152 WARNING:scrape:Not-understood crash indication 2024-02-16T22:21:30.153 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/cbodley-2024-02-15_21:52:48-rgw-wip-cbodley-testing-distro-default-smithi/7561841/teuthology.log 2024-02-16T22:21:35.369 INFO:scrape:Found 7 distinct failure reasons 2024-02-16T22:21:35.370 INFO:scrape:Failure: Command failed (workunit test cls/test_cls_rgw.sh) on smithi039 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6a4516ad3eca4b7cf7d93b64055992bfc2113285 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh' 2024-02-16T22:21:35.370 INFO:scrape:22 jobs: ['7561820', '7561808', '7561824', '7561816', '7561849', '7561851', '7561854', '7561805', '7561846', '7561831', '7561833', '7561823', '7561819', '7561801', '7561811', '7561840', '7561814', '7561837', '7561806', '7561852', '7561834', '7561845'] 2024-02-16T22:21:35.370 INFO:scrape:suites intersection: ['cls', 'clusters/fixed-2', 'frontend/beast', 'ignore-pg-availability', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'ragweed', 'reshard', 'rgw/verify/{0-install', 's3tests', 's3tests-branch', 's3tests-java', 'tasks/{bucket-check', 'versioning}'] 2024-02-16T22:21:35.370 INFO:scrape:suites union: ['cls', 'clusters/fixed-2', 'datacache/no_datacache', 'datacache/rgw-datacache', 'frontend/beast', 'ignore-pg-availability', 'inline-data$/{off}', 'inline-data$/{on}', 'mp_reupload', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'overrides', 'proto/http', 'proto/https', 'ragweed', 'reshard', 'rgw/verify/{0-install', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 's3tests', 's3tests-branch', 's3tests-java', 'sharding$/{default}', 'sharding$/{single}', 'striping$/{stripe-equals-chunk}', 'striping$/{stripe-greater-than-chunk}', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/{bucket-check', 'validater/lockdep}', 'validater/valgrind}', 'versioning}'] 2024-02-16T22:21:35.371 INFO:scrape: 2024-02-16T22:21:35.371 INFO:scrape:Failure: rgw multisite test failures 2024-02-16T22:21:35.371 INFO:scrape:2 jobs: ['7561827', '7561792'] 2024-02-16T22:21:35.371 INFO:scrape:suites intersection: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'rgw/multisite/{clusters', 'tasks/test_multi}'] 2024-02-16T22:21:35.371 INFO:scrape:suites union: ['frontend/beast', 'ignore-pg-availability', 'notify', 'omap_limits', 'overrides', 'realms/three-zones', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'supported-random-distro$/{centos_latest}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/test_multi}'] 2024-02-16T22:21:35.371 INFO:scrape: 2024-02-16T22:21:35.371 INFO:scrape:Failure: Command failed on smithi135 with status 128: 'cd /home/ubuntu/cephtest && wget http://archive.apache.org/dist/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz && tar -xvf apache-maven-3.6.3-bin.tar.gz && git clone https://github.com/apache/hadoop && cd hadoop && git checkout -b hadoop-3.2.0 rel/release-3.2.0' 2024-02-16T22:21:35.371 INFO:scrape:1 jobs: ['7561826'] 2024-02-16T22:21:35.371 INFO:scrape:suites: ['hadoop/v32', 'ignore-pg-availability', 'overrides', 'rgw/hadoop-s3a/{clusters/fixed-2', 's3a-hadoop', 'supported-random-distro$/{ubuntu_latest}}'] 2024-02-16T22:21:35.371 INFO:scrape: 2024-02-16T22:21:35.372 INFO:scrape:Failure: S3ResponseError: 403 Forbidden InvalidAccessKeyIdtx00000708e2b8cc9637974-0065cf3fcf-4225-default4225-default-default 2024-02-16T22:21:35.372 INFO:scrape:3 jobs: ['7561839', '7561795', '7561818'] 2024-02-16T22:21:35.372 INFO:scrape:suites intersection: ['frontend/beast', 'ignore-pg-availability', 'objectstore/bluestore-bitmap', 'overrides', 'rgw/singleton/{all/radosgw-admin', 'supported-random-distro$/{ubuntu_latest}}'] 2024-02-16T22:21:35.372 INFO:scrape:suites union: ['frontend/beast', 'ignore-pg-availability', 'objectstore/bluestore-bitmap', 'overrides', 'rgw/singleton/{all/radosgw-admin', 'rgw_pool_type/ec', 'rgw_pool_type/ec-profile', 'rgw_pool_type/replicated', 'supported-random-distro$/{ubuntu_latest}}'] 2024-02-16T22:21:35.372 INFO:scrape: 2024-02-16T22:21:35.372 INFO:scrape:Dead: None 2024-02-16T22:21:35.372 INFO:scrape:2 jobs: ['7561841', '7561803'] 2024-02-16T22:21:35.372 INFO:scrape: 2024-02-16T22:21:35.372 INFO:scrape:Dead: b'2024-02-16T22:17:20.373 DEBUG:teuthology.exit:Finished running handlers' 2024-02-16T22:21:35.372 INFO:scrape:1 jobs: ['7561793'] 2024-02-16T22:21:35.373 INFO:scrape:suites: ['bluestore-bitmap', 'centos_latest', 'fixed-2', 'ignore-pg-availability', 'overrides', 'rgw/notifications/{beast', 'tasks/{0-install', 'test_amqp', 'test_kafka', 'test_others}}'] 2024-02-16T22:21:35.373 INFO:scrape: 2024-02-16T22:21:35.373 INFO:scrape:Failure: [Errno None] Unable to connect to port 22 on 172.21.15.112 2024-02-16T22:21:35.373 INFO:scrape:1 jobs: ['7561842'] 2024-02-16T22:21:35.373 INFO:scrape:suites: ['frontend/beast', 'ignore-pg-availability', 'objectstore/bluestore-bitmap', 'overrides', 'rgw/multifs/{clusters/fixed-2', 'rgw_pool_type/replicated', 's3tests-branch', 'tasks/rgw_multipart_upload', 'ubuntu_latest}'] 2024-02-16T22:21:35.373 INFO:scrape: