2022-02-13T17:39:54.570 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-13T17:39:54.572 INFO:teuthology.results:subset: '111/120000' 2022-02-13T17:39:54.572 INFO:teuthology.results:seed: '4543' 2022-02-13T18:02:18.217 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-13T18:02:18.243 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-02-13T20:35:29.522 INFO:teuthology.results:Tests finished! gathering results... 2022-02-13T20:35:33.011 INFO:scrape:Found 39 jobs 2022-02-13T20:35:34.256 WARNING:scrape:Not-understood crash indication 2022-02-13T18:21:16.956 INFO:tasks.rgw.c2.client.0.smithi122.stderr:daemon-helper: command crashed with signal 11 2022-02-13T20:35:34.257 WARNING:scrape:Not-understood crash indication 2022-02-13T18:21:16.981 INFO:tasks.rgw.c1.client.1.smithi026.stderr:daemon-helper: command crashed with signal 11 2022-02-13T20:35:34.257 WARNING:scrape:Not-understood crash indication 2022-02-13T20:35:34.283 WARNING:scrape:Not-understood crash indication 2022-02-13T18:21:16.956 INFO:tasks.rgw.c2.client.0.smithi122.stderr:daemon-helper: command crashed with signal 11 2022-02-13T20:35:34.284 WARNING:scrape:Not-understood crash indication 2022-02-13T18:21:16.981 INFO:tasks.rgw.c1.client.1.smithi026.stderr:daemon-helper: command crashed with signal 11 2022-02-13T20:35:34.285 WARNING:scrape:Not-understood crash indication 2022-02-13T20:35:34.748 WARNING:scrape:Not-understood crash indication 2022-02-13T18:36:11.497 INFO:tasks.rgw.c1.client.1.smithi131.stderr:daemon-helper: command crashed with signal 11 2022-02-13T20:35:34.749 WARNING:scrape:Not-understood crash indication 2022-02-13T18:36:11.506 INFO:tasks.rgw.c2.client.0.smithi166.stderr:daemon-helper: command crashed with signal 11 2022-02-13T20:35:34.750 WARNING:scrape:Not-understood crash indication 2022-02-13T20:35:34.776 WARNING:scrape:Not-understood crash indication 2022-02-13T18:36:11.497 INFO:tasks.rgw.c1.client.1.smithi131.stderr:daemon-helper: command crashed with signal 11 2022-02-13T20:35:34.777 WARNING:scrape:Not-understood crash indication 2022-02-13T18:36:11.506 INFO:tasks.rgw.c2.client.0.smithi166.stderr:daemon-helper: command crashed with signal 11 2022-02-13T20:35:34.777 WARNING:scrape:Not-understood crash indication 2022-02-13T20:35:35.049 INFO:scrape:Found 4 distinct failure reasons 2022-02-13T20:35:35.049 INFO:scrape:Failure: Command failed on smithi016 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' 2022-02-13T20:35:35.050 INFO:scrape:1 jobs: ['6680506'] 2022-02-13T20:35:35.050 INFO:scrape:suites: ['frontend/civetweb', 'objectstore/filestore-xfs', 'overrides', 'rgw/multifs/{clusters/fixed-2', 'rgw_pool_type/ec', 'tasks/rgw_ragweed}'] 2022-02-13T20:35:35.051 INFO:scrape: 2022-02-13T20:35:35.051 INFO:scrape:Failure: rgw multisite, test failures 2022-02-13T20:35:35.052 INFO:scrape:3 jobs: ['6680528', '6680498', '6680518'] 2022-02-13T20:35:35.052 INFO:scrape:suites intersection: ['omap_limits', 'overrides', 'rgw/multisite/{clusters', 'tasks/test_multi', 'valgrind}'] 2022-02-13T20:35:35.052 INFO:scrape:suites union: ['frontend/beast', 'frontend/civetweb', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'tasks/test_multi', 'valgrind}'] 2022-02-13T20:35:35.053 INFO:scrape: 2022-02-13T20:35:35.053 INFO:scrape:Failure: Command failed on smithi082 with status 4: 'cd /home/ubuntu/cephtest && wget http://www-us.apache.org/dist/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz && tar -xvf apache-maven-3.6.3-bin.tar.gz && git clone https://github.com/apache/hadoop && cd hadoop && git checkout -b hadoop-3.2.0 rel/release-3.2.0' 2022-02-13T20:35:35.054 INFO:scrape:2 jobs: ['6680516', '6680496'] 2022-02-13T20:35:35.054 INFO:scrape:suites intersection: ['overrides', 'rgw/hadoop-s3a/{clusters/fixed-2', 's3a-hadoop}'] 2022-02-13T20:35:35.055 INFO:scrape:suites union: ['hadoop/default', 'hadoop/v32', 'overrides', 'rgw/hadoop-s3a/{clusters/fixed-2', 's3a-hadoop}'] 2022-02-13T20:35:35.055 INFO:scrape: 2022-02-13T20:35:35.056 INFO:scrape:Failure: Cannot create secret 2022-02-13T20:35:35.056 INFO:scrape:1 jobs: ['6680495'] 2022-02-13T20:35:35.056 INFO:scrape:suites: ['1-ceph-install/install', '2-kms/barbican', '3-rgw/rgw', '4-tests/{s3tests}}', 'rgw/crypt/{0-cluster/fixed-1'] 2022-02-13T20:35:35.057 INFO:scrape: 2022-02-13T20:35:35.059 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/ failed: 7 dead: 0 running: 0 waiting: 0 queued: 0 passed: 32 Fail ================================================================= [6680495] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests}} ----------------------------------------------------------------- time: 00:13:30 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680495/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680495/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0967a3d84980463496aba78c1bc84f6f Cannot create secret [6680496] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} ----------------------------------------------------------------- time: 00:09:16 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680496/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680496/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bc113d5e757c40439b6eef311e044f76 Command failed on smithi132 with status 4: 'cd /home/ubuntu/cephtest && wget http://www-us.apache.org/dist/maven/maven-3/3.6.3/binaries/apache- maven-3.6.3-bin.tar.gz && tar -xvf apache-maven-3.6.3-bin.tar.gz && git clone https://github.com/apache/hadoop && cd hadoop && git checkout -b hadoop-2.9.2 rel/release-2.9.2' [6680498] rgw/multisite/{clusters frontend/beast omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:46:24 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680498/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680498/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b73db71023d44ec58dad1c45fd56ecb4 rgw multisite, rgw multisite pubsub, test failures [6680506] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_ragweed} ----------------------------------------------------------------- time: 00:06:24 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680506/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680506/ sentry: https://sentry.ceph.com/organizations/ceph/?query=040470bd2cf0411da0bc133a7c0ad5d6 Command failed on smithi016 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [6680516] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} ----------------------------------------------------------------- time: 00:08:26 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680516/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680516/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2bb94973ee144dc68fd7745f3d52db4a Command failed on smithi082 with status 4: 'cd /home/ubuntu/cephtest && wget http://www-us.apache.org/dist/maven/maven-3/3.6.3/binaries/apache- maven-3.6.3-bin.tar.gz && tar -xvf apache-maven-3.6.3-bin.tar.gz && git clone https://github.com/apache/hadoop && cd hadoop && git checkout -b hadoop-3.2.0 rel/release-3.2.0' [6680518] rgw/multisite/{clusters frontend/civetweb omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:48:33 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680518/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680518/ sentry: https://sentry.ceph.com/organizations/ceph/?query=811c29cb0d6445e1b49f288aaefa5c7d rgw multisite, rgw multisite pubsub, test failures [6680528] rgw/multisite/{clusters frontend/beast omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:20:37 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680528/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680528/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2642358269b2479696e42ce8dde20dcd rgw multisite, test failures Pass ================================================================= [6680497] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload} time: 00:08:07 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680497/ [6680499] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:19:24 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680499/ [6680500] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} time: 00:19:52 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680500/ [6680501] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_bucket_quota} time: 00:12:07 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680501/ [6680502] rgw/tools/{centos_latest cluster tasks} time: 02:41:58 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680502/ [6680503] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed s3tests-java s3tests} validater/lockdep} time: 00:21:34 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680503/ [6680504] rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest} time: 00:14:10 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680504/ [6680505] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_multipart_upload} time: 00:10:16 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680505/ [6680507] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_18.04}} time: 00:09:19 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680507/ [6680508] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests}} time: 00:09:49 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680508/ [6680509] rgw/multisite/{clusters frontend/civetweb omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} time: 00:20:43 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680509/ [6680510] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_s3tests} time: 00:15:55 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680510/ [6680511] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed s3tests-java s3tests} validater/valgrind} time: 00:26:11 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680511/ [6680512] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_s3tests} time: 00:11:13 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680512/ [6680513] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_18.04}} time: 00:09:30 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680513/ [6680514] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:10:53 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680514/ [6680515] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests}} time: 00:09:54 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680515/ [6680517] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_user_quota} time: 00:07:48 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680517/ [6680519] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:19:19 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680519/ [6680520] rgw/tempest/{clusters/fixed-1 frontend/civetweb overrides tasks/rgw_tempest ubuntu_latest} time: 00:20:43 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680520/ [6680521] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_bucket_quota} time: 00:11:47 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680521/ [6680522] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed s3tests-java s3tests} validater/lockdep} time: 00:23:24 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680522/ [6680523] rgw/website/{clusters/fixed-2 frontend/civetweb http overrides tasks/s3tests-website ubuntu_latest} time: 00:14:40 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680523/ [6680524] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_multipart_upload} time: 00:10:46 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680524/ [6680525] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_bucket_quota} time: 00:08:46 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680525/ [6680526] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{rhel_8}} time: 00:19:35 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680526/ [6680527] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests}} time: 00:10:22 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680527/ [6680529] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_s3tests} time: 00:14:18 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680529/ [6680530] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed s3tests-java s3tests} validater/valgrind} time: 00:26:33 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680530/ [6680531] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload} time: 00:07:14 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680531/ [6680532] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_18.04}} time: 00:09:27 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680532/ [6680533] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:10:40 info: http://pulpito.ceph.com/yuriw-2022-02-13_15:59:35-rgw-octopus-distro-default-smithi/6680533/