2022-03-24T03:10:19.135 INFO:root:teuthology version: 1.1.0-a4e18662 2022-03-24T03:10:19.136 INFO:teuthology.results:subset: '111/120000' 2022-03-24T03:10:19.136 INFO:teuthology.results:seed: '94' 2022-03-24T03:51:07.193 INFO:root:teuthology version: 1.1.0-a4e18662 2022-03-24T03:51:07.198 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-03-24T10:28:36.646 INFO:teuthology.results:Tests finished! gathering results... 2022-03-24T10:28:39.554 INFO:scrape:Found 43 jobs 2022-03-24T10:28:39.795 WARNING:scrape:Not-understood crash indication 2022-03-24T04:06:25.569 INFO:tasks.rgw.c1.client.0.smithi029.stderr:daemon-helper: command crashed with signal 11 2022-03-24T10:28:39.796 WARNING:scrape:Not-understood crash indication 2022-03-24T04:06:29.479 INFO:tasks.rgw.c1.client.1.smithi029.stderr:daemon-helper: command crashed with signal 11 2022-03-24T10:28:39.796 WARNING:scrape:Not-understood crash indication 2022-03-24T10:28:40.036 INFO:scrape:Found 3 distinct failure reasons 2022-03-24T10:28:40.037 INFO:scrape:Failure: Command failed on smithi122 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' 2022-03-24T10:28:40.038 INFO:scrape:1 jobs: ['6756540'] 2022-03-24T10:28:40.038 INFO:scrape:suites: ['frontend/civetweb', 'ignore-pg-availability', 'objectstore/filestore-xfs', 'overrides', 'rgw/multifs/{clusters/fixed-2', 'rgw_pool_type/ec', 'tasks/rgw_ragweed}'] 2022-03-24T10:28:40.039 INFO:scrape: 2022-03-24T10:28:40.039 INFO:scrape:Dead: b'2022-03-24T10:23:26.730 DEBUG:teuthology.exit:Finished running handlers' 2022-03-24T10:28:40.040 INFO:scrape:4 jobs: ['6756563', '6756530', '6756552', '6756542'] 2022-03-24T10:28:40.040 INFO:scrape:suites intersection: ['ignore-pg-availability', 'omap_limits', 'overrides', 'rgw/multisite/{clusters', 'tasks/test_multi', 'valgrind}'] 2022-03-24T10:28:40.041 INFO:scrape:suites union: ['frontend/beast', 'frontend/civetweb', 'ignore-pg-availability', 'omap_limits', 'overrides', 'realms/three-zone-plus-pubsub', 'realms/two-zonegroup', 'rgw/multisite/{clusters', 'tasks/test_multi', 'valgrind}'] 2022-03-24T10:28:40.041 INFO:scrape: 2022-03-24T10:28:40.042 INFO:scrape:Failure: Cannot create secret 2022-03-24T10:28:40.042 INFO:scrape:1 jobs: ['6756527'] 2022-03-24T10:28:40.043 INFO:scrape:suites: ['1-ceph-install/install', '2-kms/barbican', '3-rgw/rgw', '4-tests/{s3tests}', 'ignore-pg-availability}', 'rgw/crypt/{0-cluster/fixed-1'] 2022-03-24T10:28:40.043 INFO:scrape: 2022-03-24T10:28:40.045 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/ failed: 2 dead: 4 running: 0 waiting: 0 queued: 0 passed: 37 Fail ================================================================= [6756527] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/barbican 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} ----------------------------------------------------------------- time: 00:15:39 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756527/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756527/ sentry: https://sentry.ceph.com/organizations/ceph/?query=610cfa15033d4687b073714dc435b276 Cannot create secret [6756540] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec tasks/rgw_ragweed} ----------------------------------------------------------------- time: 00:06:41 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756540/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756540/ sentry: https://sentry.ceph.com/organizations/ceph/?query=152585ea2f6a4955a1cce2a4ade36a2f Command failed on smithi122 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' Dead ================================================================= [6756530] rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756530/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756530/ hit max job timeout [6756542] rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756542/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756542/ hit max job timeout [6756552] rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756552/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756552/ hit max job timeout [6756563] rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/two-zonegroup tasks/test_multi valgrind} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756563/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756563/ hit max job timeout Pass ================================================================= [6756528] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/default overrides s3a-hadoop} time: 00:14:51 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756528/ [6756529] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile tasks/rgw_multipart_upload} time: 00:08:17 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756529/ [6756531] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_latest}} time: 00:09:31 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756531/ [6756532] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/beast supported-random-distro$/{centos_8} tasks/{0-install ststests webidentity}} time: 00:27:51 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756532/ [6756533] rgw/tempest/{clusters/fixed-1 frontend/beast overrides tasks/rgw_tempest ubuntu_latest} time: 00:21:23 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756533/ [6756534] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_bucket_quota} time: 00:11:57 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756534/ [6756535] rgw/tools/{centos_latest cluster tasks} time: 02:47:58 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756535/ [6756536] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/ec-profile sharding$/{default} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:25:45 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756536/ [6756537] rgw/website/{clusters/fixed-2 frontend/beast http overrides tasks/s3tests-website ubuntu_latest} time: 00:15:22 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756537/ [6756538] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_multipart_upload} time: 00:10:03 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756538/ [6756539] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/kmip 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:11:18 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756539/ [6756541] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec supported-random-distro$/{rhel_8}} time: 00:18:26 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756541/ [6756543] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_s3tests} time: 00:15:12 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756543/ [6756544] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 00:30:18 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756544/ [6756545] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/testing 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:10:30 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756545/ [6756546] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated tasks/rgw_s3tests} time: 00:12:02 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756546/ [6756547] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{ubuntu_latest}} time: 00:09:23 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756547/ [6756548] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:10:56 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756548/ [6756549] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_kv 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:10:39 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756549/ [6756550] rgw/hadoop-s3a/{clusters/fixed-2 hadoop/v32 overrides s3a-hadoop} time: 00:13:37 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756550/ [6756551] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/ec-profile tasks/rgw_user_quota} time: 00:07:32 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756551/ [6756553] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/ec-profile supported-random-distro$/{rhel_8}} time: 00:19:13 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756553/ [6756554] rgw/sts/{cluster ignore-pg-availability objectstore overrides pool-type rgw_frontend/civetweb supported-random-distro$/{rhel_8} tasks/{0-install ststests webidentity}} time: 00:34:53 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756554/ [6756555] rgw/tempest/{clusters/fixed-1 frontend/civetweb overrides tasks/rgw_tempest ubuntu_latest} time: 00:20:51 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756555/ [6756556] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_bucket_quota} time: 00:11:50 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756556/ [6756557] rgw/verify/{centos_latest clusters/fixed-2 frontend/beast ignore-pg-availability msgr-failures/few objectstore/filestore-xfs overrides proto/http rgw_pool_type/replicated sharding$/{single} striping$/{stripe-greater-than-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/lockdep} time: 00:26:08 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756557/ [6756558] rgw/website/{clusters/fixed-2 frontend/civetweb http overrides tasks/s3tests-website ubuntu_latest} time: 00:15:06 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756558/ [6756559] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_multipart_upload} time: 00:11:13 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756559/ [6756560] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_old 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:10:45 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756560/ [6756561] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec tasks/rgw_bucket_quota} time: 00:08:50 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756561/ [6756562] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{ubuntu_latest}} time: 00:09:23 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756562/ [6756564] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/filestore-xfs thrasher/default thrashosds-health workload/rgw_s3tests} time: 00:17:02 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756564/ [6756565] rgw/verify/{centos_latest clusters/fixed-2 frontend/civetweb ignore-pg-availability msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec-profile sharding$/{single} striping$/{stripe-equals-chunk} tasks/{0-install cls ragweed reshard s3tests-java s3tests} validater/valgrind} time: 00:29:51 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756565/ [6756566] rgw/crypt/{0-cluster/fixed-1 1-ceph-install/install 2-kms/vault_transit 3-rgw/rgw 4-tests/{s3tests} ignore-pg-availability} time: 00:10:41 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756566/ [6756567] rgw/multifs/{clusters/fixed-2 frontend/civetweb ignore-pg-availability objectstore/filestore-xfs overrides rgw_pool_type/replicated tasks/rgw_multipart_upload} time: 00:07:59 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756567/ [6756568] rgw/singleton/{all/radosgw-admin frontend/civetweb objectstore/filestore-xfs overrides rgw_pool_type/replicated supported-random-distro$/{rhel_8}} time: 00:18:19 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756568/ [6756569] rgw/thrash/{civetweb clusters/fixed-2 install objectstore/bluestore-bitmap thrasher/default thrashosds-health workload/rgw_user_quota} time: 00:11:28 info: http://pulpito.ceph.com/yuriw-2022-03-24_01:56:50-rgw-wip-yuri4-testing-2022-03-23-1127-pacific-distro-default-smithi/6756569/