2020-12-09T18:02:33.756 INFO:root:teuthology version: 1.0.0-c0cbad1b 2020-12-09T18:02:33.757 INFO:teuthology.results:subset: '1/20' 2020-12-09T18:02:33.757 INFO:teuthology.results:seed: '831' 2020-12-09T18:44:55.174 INFO:root:teuthology version: 1.0.0-c0cbad1b 2020-12-09T18:44:55.211 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-12-09T20:18:57.020 INFO:teuthology.results:Tests finished! gathering results... 2020-12-09T20:19:01.018 INFO:scrape:Found 43 jobs 2020-12-09T20:19:02.158 INFO:scrape:Found 3 distinct failure reasons 2020-12-09T20:19:02.159 INFO:scrape:Failure: Command failed on smithi027 with status 1: 'cd /home/ubuntu/cephtest/keystone && cp -f etc/keystone.conf.sample etc/keystone.conf' 2020-12-09T20:19:02.160 INFO:scrape:2 jobs: ['5696758', '5696780'] 2020-12-09T20:19:02.160 INFO:scrape:suites intersection: ['rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}'] 2020-12-09T20:19:02.160 INFO:scrape:suites union: ['frontend/beast', 'frontend/civetweb', 'rgw/tempest/{clusters/fixed-1', 'tasks/rgw_tempest', 'ubuntu_latest}'] 2020-12-09T20:19:02.161 INFO:scrape: 2020-12-09T20:19:02.161 INFO:scrape:Failure: rgw multisite, test failures 2020-12-09T20:19:02.162 INFO:scrape:2 jobs: ['5696756', '5696778'] 2020-12-09T20:19:02.162 INFO:scrape:suites intersection: ['omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'rgw/multisite/{clusters', 'tasks/test_multi', 'valgrind}'] 2020-12-09T20:19:02.162 INFO:scrape:suites union: ['frontend/beast', 'frontend/civetweb', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'rgw/multisite/{clusters', 'tasks/test_multi', 'valgrind}'] 2020-12-09T20:19:02.163 INFO:scrape: 2020-12-09T20:19:02.163 INFO:scrape:Failure: reached maximum tries (100) after waiting for 600 seconds 2020-12-09T20:19:02.164 INFO:scrape:1 jobs: ['5696785'] 2020-12-09T20:19:02.164 INFO:scrape: 2020-12-09T20:19:02.166 INFO:teuthology.results:Sending results to ivancich@redhat.com: Test Run: ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/ failed: 4 dead: 1 running: 0 waiting: 0 queued: 0 passed: 38 Fail ================================================================= [5696756] rgw/multisite/{clusters frontend/beast omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} ----------------------------------------------------------------- time: 01:40:01 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696756/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696756/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e58c4a3b37dc4602842d272365668d84 rgw multisite, test failures [5696758] rgw/tempest/{clusters/fixed-1 frontend/beast tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:10:29 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696758/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696758/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4c263770ca0047ceb0a681ac0d196fc1 Command failed on smithi027 with status 1: 'cd /home/ubuntu/cephtest/keystone && cp -f etc/keystone.conf.sample etc/keystone.conf' [5696778] rgw/multisite/{clusters frontend/civetweb omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} ----------------------------------------------------------------- time: 01:40:53 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696778/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696778/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bac4e7a6f83344dbb0132d62e70efda5 rgw multisite, test failures [5696780] rgw/tempest/{clusters/fixed-1 frontend/civetweb tasks/rgw_tempest ubuntu_latest} ----------------------------------------------------------------- time: 00:10:33 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696780/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696780/ sentry: https://sentry.ceph.com/organizations/ceph/?query=129856eb52154159bb283b1715710771 Command failed on smithi035 with status 1: 'cd /home/ubuntu/cephtest/keystone && cp -f etc/keystone.conf.sample etc/keystone.conf' Dead ================================================================= [5696785] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-stupid overrides rgw_pool_type/ec tasks/rgw_bucket_quota} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696785/ log: http://qa-proxy.ceph.com/teuthology/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696785/ sentry: https://sentry.ceph.com/organizations/ceph/?query=129aa167e7194a3c974295117593a453 reached maximum tries (100) after waiting for 600 seconds Pass ================================================================= [5696754] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-comp-zlib thrasher/default thrashosds-health workload/rgw_s3tests} time: 00:46:13 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696754/ [5696755] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default s3a-hadoop supported-random-distro$/{ubuntu_16.04}} time: 00:14:29 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696755/ [5696757] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-comp-zstd overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_latest}} time: 00:08:49 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696757/ [5696759] rgw/tools/{centos_latest cluster tasks} time: 00:21:50 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696759/ [5696760] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-comp-lz4 overrides rgw_pool_type/ec tasks/rgw_ragweed} time: 00:08:20 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696760/ [5696761] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-comp-zstd thrasher/default thrashosds-health workload/rgw_swift} time: 00:10:27 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696761/ [5696762] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-hybrid overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}} time: 00:11:57 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696762/ [5696763] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-comp-snappy overrides rgw_pool_type/replicated tasks/rgw_readwrite} time: 00:14:28 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696763/ [5696764] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-hybrid thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:11:36 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696764/ [5696765] rgw/multisite/{clusters frontend/civetweb omap_limits overrides realms/three-zone tasks/test_multi valgrind} time: 00:29:18 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696765/ [5696766] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-stupid overrides rgw_pool_type/replicated supported-random-distro$/{rhel_7}} time: 00:12:20 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696766/ [5696767] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-comp-zlib overrides rgw_pool_type/ec-profile tasks/rgw_roundtrip} time: 00:06:37 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696767/ [5696768] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_7}} time: 00:12:16 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696768/ [5696769] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-stupid thrasher/default thrashosds-health workload/rgw_bucket_quota} time: 00:11:52 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696769/ [5696770] rgw/multisite/{clusters frontend/beast omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} time: 00:21:56 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696770/ [5696771] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}} time: 00:12:23 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696771/ [5696772] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_multipart_upload} time: 00:10:30 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696772/ [5696773] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-comp-zstd overrides rgw_pool_type/ec tasks/rgw_s3tests} time: 00:40:28 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696773/ [5696774] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-comp-lz4 overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} time: 00:08:34 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696774/ [5696775] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_readwrite} time: 00:17:59 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696775/ [5696776] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-hybrid overrides rgw_pool_type/replicated tasks/rgw_swift} time: 00:06:57 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696776/ [5696777] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 s3a-hadoop supported-random-distro$/{ubuntu_latest}} time: 00:12:55 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696777/ [5696779] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-comp-snappy overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_latest}} time: 00:11:59 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696779/ [5696781] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-comp-lz4 thrasher/default thrashosds-health workload/rgw_roundtrip} time: 00:09:39 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696781/ [5696782] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-stupid overrides rgw_pool_type/ec-profile tasks/rgw_user_quota} time: 00:07:30 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696782/ [5696783] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-comp-zlib overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}} time: 00:12:24 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696783/ [5696784] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-comp-snappy thrasher/default thrashosds-health workload/rgw_s3tests} time: 00:45:36 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696784/ [5696786] rgw/multisite/{clusters frontend/beast omap_limits overrides realms/three-zone tasks/test_multi valgrind} time: 00:27:32 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696786/ [5696787] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-comp-zstd overrides rgw_pool_type/replicated supported-random-distro$/{rhel_7}} time: 00:12:28 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696787/ [5696788] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-comp-zlib thrasher/default thrashosds-health workload/rgw_swift} time: 00:10:32 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696788/ [5696789] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-hybrid overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_16.04}} time: 00:09:12 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696789/ [5696790] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload} time: 00:06:53 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696790/ [5696791] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-comp-zstd thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:11:18 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696791/ [5696792] rgw/multisite/{clusters frontend/civetweb omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} time: 00:21:53 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696792/ [5696793] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-stupid overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_latest}} time: 00:08:56 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696793/ [5696794] rgw/multifs/{clusters/fixed-2 frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_ragweed} time: 00:08:48 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696794/ [5696795] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-hybrid thrasher/default thrashosds-health workload/rgw_bucket_quota} time: 00:12:46 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696795/ [5696796] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{centos_latest}} time: 00:12:11 info: http://pulpito.ceph.com/ivancich-2020-12-09_16:52:34-rgw-nautilus-better-ordered-listing-skipping-distro-basic-smithi/5696796/