2021-07-02T22:41:08.935 INFO:root:teuthology version: 1.1.0-26631ede 2021-07-02T22:41:08.936 INFO:teuthology.results:subset: '111/120000' 2021-07-02T22:41:08.937 INFO:teuthology.results:seed: '1436' 2021-07-02T22:42:10.581 INFO:root:teuthology version: 1.1.0-26631ede 2021-07-02T22:42:10.583 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-07-03T01:41:25.196 INFO:teuthology.results:Tests finished! gathering results... 2021-07-03T01:41:29.570 INFO:scrape:Found 48 jobs 2021-07-03T01:41:29.795 WARNING:scrape:Not-understood crash indication 2021-07-02T23:06:36.983 INFO:tasks.rgw.client.1.smithi049.stderr:daemon-helper: command crashed with signal 11 2021-07-03T01:41:29.796 WARNING:scrape:Not-understood crash indication 2021-07-03T01:41:29.862 WARNING:scrape:Not-understood crash indication 2021-07-02T23:06:36.983 INFO:tasks.rgw.client.1.smithi049.stderr:daemon-helper: command crashed with signal 11 2021-07-03T01:41:29.864 WARNING:scrape:Not-understood crash indication 2021-07-03T01:41:29.939 WARNING:scrape:Not-understood crash indication 2021-07-02T23:06:36.983 INFO:tasks.rgw.client.1.smithi049.stderr:daemon-helper: command crashed with signal 11 2021-07-03T01:41:29.941 WARNING:scrape:Not-understood crash indication 2021-07-03T01:41:30.293 WARNING:scrape:Not-understood crash indication 2021-07-02T23:07:22.409 INFO:tasks.rgw.client.1.smithi142.stderr:daemon-helper: command crashed with signal 11 2021-07-03T01:41:30.294 WARNING:scrape:Not-understood crash indication 2021-07-03T01:41:30.295 INFO:scrape:Found 2 distinct failure reasons 2021-07-03T01:41:30.296 INFO:scrape:Failure: Command failed (s3 tests against rgw) on smithi025 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!s3select,!test_of_sts,!webidentity_test,!fails_with_subdomain'" 2021-07-03T01:41:30.296 INFO:scrape:2 jobs: ['6250290', '6250267'] 2021-07-03T01:41:30.297 INFO:scrape:suites intersection: ['http', 'overrides', 'rgw/website/{clusters/fixed-2', 'tasks/s3tests-website', 'ubuntu_latest}'] 2021-07-03T01:41:30.297 INFO:scrape:suites union: ['frontend/beast', 'frontend/civetweb', 'http', 'overrides', 'rgw/website/{clusters/fixed-2', 'tasks/s3tests-website', 'ubuntu_latest}'] 2021-07-03T01:41:30.297 INFO:scrape: 2021-07-03T01:41:30.298 INFO:scrape:Failure: list index out of range 2021-07-03T01:41:30.298 INFO:scrape:1 jobs: ['6250258'] 2021-07-03T01:41:30.299 INFO:scrape:suites: ['rgw/ignore-pg-availability'] 2021-07-03T01:41:30.300 INFO:scrape: 2021-07-03T01:41:30.302 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/ failed: 3 dead: 0 running: 0 waiting: 0 queued: 0 passed: 45 Fail ================================================================= [6250258] rgw/ignore-pg-availability ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250258/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250258/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b240176b131447f495890cf0a0fe3918 list index out of range [6250267] rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest} ----------------------------------------------------------------- time: 00:18:40 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250267/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250267/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e8fcfc238afe42e08dc7eaf408b51352 Command failed (s3 tests against rgw) on smithi098 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!s3select,!test_of_sts,!webidentity_test,!fails_with_s ubdomain'" [6250290] rgw/website/{clusters/fixed-2 frontend/civetweb http overrides tasks/s3tests-website ubuntu_latest} ----------------------------------------------------------------- time: 00:19:45 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250290/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250290/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7c043d4973604381a68a02dd34a527de Command failed (s3 tests against rgw) on smithi025 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python -m nose -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,! fails_strict_rfc2616,!s3select,!test_of_sts,!webidentity_test,!fails_with_s ubdomain'" Pass ================================================================= [6250256] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:21:18 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250256/ [6250257] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} time: 00:18:19 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250257/ [6250259] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload} time: 00:15:28 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250259/ [6250260] rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} time: 00:32:26 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250260/ [6250261] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:20:28 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250261/ [6250262] rgw/sts/{centos_latest clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/{0-install ststests webidentity}} time: 00:33:04 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250262/ [6250263] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} time: 00:22:51 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250263/ [6250264] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_bucket_quota} time: 00:16:25 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250264/ [6250265] rgw/tools/{centos_latest cluster tasks} time: 02:51:29 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250265/ [6250266] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:29:27 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250266/ [6250268] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_multipart_upload} time: 00:14:08 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250268/ [6250269] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:14:11 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250269/ [6250270] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_ragweed} time: 00:15:07 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250270/ [6250271] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_latest}} time: 00:13:45 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250271/ [6250272] rgw/sts/{centos_latest clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/{0-install ststests webidentity}} time: 00:30:34 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250272/ [6250273] rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} time: 00:29:45 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250273/ [6250274] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_s3tests} time: 00:23:21 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250274/ [6250275] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 00:36:44 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250275/ [6250276] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:14:30 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250276/ [6250277] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_s3tests} time: 00:20:00 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250277/ [6250278] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} time: 00:13:16 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250278/ [6250279] rgw/sts/{centos_latest clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/{0-install ststests webidentity}} time: 00:32:52 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250279/ [6250280] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:16:05 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250280/ [6250281] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:14:44 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250281/ [6250282] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} time: 00:17:57 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250282/ [6250283] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_user_quota} time: 00:15:35 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250283/ [6250284] rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} time: 00:33:30 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250284/ [6250285] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:25:07 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250285/ [6250286] rgw/sts/{centos_latest clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/{0-install ststests webidentity}} time: 00:31:05 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250286/ [6250287] rgw/tempest/{clusters/fixed-1 frontend/civetweb overrides tasks/rgw_tempest ubuntu_latest} time: 00:22:54 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250287/ [6250288] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_bucket_quota} time: 00:18:50 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250288/ [6250289] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:29:18 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250289/ [6250291] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_multipart_upload} time: 00:20:07 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250291/ [6250292] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:14:01 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250292/ [6250293] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_bucket_quota} time: 00:12:06 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250293/ [6250294] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_8}} time: 00:14:59 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250294/ [6250295] rgw/sts/{centos_latest clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/{0-install ststests webidentity}} time: 00:33:22 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250295/ [6250296] rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} time: 00:26:34 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250296/ [6250297] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_s3tests} time: 00:24:20 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250297/ [6250298] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 00:35:35 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250298/ [6250299] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:15:02 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250299/ [6250300] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload} time: 00:15:20 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250300/ [6250301] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{centos_8}} time: 00:16:24 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250301/ [6250302] rgw/sts/{centos_latest clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/{0-install ststests webidentity}} time: 00:32:50 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250302/ [6250303] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:19:26 info: http://pulpito.ceph.com/yuriw-2021-07-02_22:40:43-rgw-pacific-distro-basic-smithi/6250303/