2020-05-25T05:30:51.233 INFO:root:teuthology version: 1.0.0-4413d31 2020-05-25T05:30:51.235 INFO:teuthology.results:subset: '1234/9999' 2020-05-25T05:30:51.236 INFO:teuthology.results:seed: '2477' 2020-05-25T05:48:31.294 INFO:root:teuthology version: 1.0.0-4413d31 2020-05-25T05:48:31.314 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-05-25T07:40:33.941 INFO:teuthology.results:Tests finished! gathering results... 2020-05-25T07:40:34.010 INFO:teuthology.results:Sending results to tchaikov@gmail.com: Test Run: kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/ failed: 21 dead: 0 running: 0 waiting: 0 queued: 0 passed: 1 Fail ================================================================= [5090963] rgw/verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests.yaml swift.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:13:31 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090963/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090963/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=0de5d9a195c143af914d7d416b450b9a Command failed on smithi101 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5090964] rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/bluestore-stupid.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_readwrite.yaml} ----------------------------------------------------------------- time: 00:12:35 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090964/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090964/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=0684e1ebcdc04714b7f77dc3647442b8 Command failed on smithi039 with status 2: 'cd /home/ubuntu/cephtest/s3-tests && ./bootstrap' [5090965] rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-bitmap.yaml overrides.yaml rgw_pool_type/replicated.yaml tasks/rgw_ragweed.yaml} ----------------------------------------------------------------- time: 00:07:58 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090965/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090965/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=4a13b7662f1a4b209a77226b3b1813a8 Command failed on smithi082 with status 128: 'git clone -b ceph-nautilus- ragweed-setuptools git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed' [5090966] rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-comp-lz4.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:13:05 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090966/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090966/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=9a2a3e9519e946878acaf2e203c5a226 Command failed on smithi165 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5090967] rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/filestore-xfs.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_roundtrip.yaml} ----------------------------------------------------------------- time: 00:12:10 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090967/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090967/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=06e067453b7c46f29fc89021589ea71d Command failed on smithi191 with status 2: 'cd /home/ubuntu/cephtest/s3-tests && ./bootstrap' [5090968] rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-comp-lz4.yaml overrides.yaml rgw_pool_type/ec-profile.yaml tasks/rgw_readwrite.yaml} ----------------------------------------------------------------- time: 00:08:15 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090968/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090968/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=0af3d5b77bd84e3e943dfc5eec0ab2bb Command failed on smithi073 with status 2: 'cd /home/ubuntu/cephtest/s3-tests && ./bootstrap' [5090969] rgw/hadoop-s3a/{clusters/fixed-2.yaml hadoop/v28.yaml s3a-hadoop.yaml supported-random-distro$/{centos_latest.yaml}} ----------------------------------------------------------------- time: 00:23:57 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090969/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090969/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=be340a94b00144ffb787f45b1edac4cb Command failed on smithi193 with status 1: 'cd /home/ubuntu/cephtest/hadoop/hadoop-tools/hadoop-aws/ && rm src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java && /home/ubuntu/cephtest/apache-maven-3.3.9/bin/mvn -Dit.test=ITestS3A* -Dparallel-tests -Dscale -Dfs.s3a.scale.test.timeout=1200 -Dfs.s3a.scale.test.huge.filesize=256M verify' [5090970] rgw/multisite/{clusters.yaml frontend/civetweb.yaml omap_limits.yaml overrides.yaml realms/three-zone-plus-pubsub.yaml tasks/test_multi.yaml valgrind.yaml} ----------------------------------------------------------------- time: 01:43:14 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090970/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090970/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=3f79b05325c7455ba0191828ce18c2dd rgw multisite, test failures [5090971] rgw/tempest/{clusters/fixed-1.yaml frontend/civetweb.yaml tasks/rgw_tempest.yaml ubuntu_latest.yaml} ----------------------------------------------------------------- time: 00:11:01 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090971/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090971/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=728bd5a42ee948deafee72560fbb263b Command failed on smithi099 with status 1: "cd /home/ubuntu/cephtest/keystone && source .tox/venv/bin/activate && openstack project create --os-token ADMIN --os-url http://smithi099.front.sepia.ceph.com:35357/v2.0 --description 'Admin Tenant' admin" [5090972] rgw/verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-comp-snappy.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec.yaml striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests.yaml swift.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:13:59 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090972/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090972/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=bb0b13b45bea46908a0843a30bedc675 Command failed on smithi109 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5090973] rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/bluestore-bitmap.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_s3tests.yaml} ----------------------------------------------------------------- time: 00:11:36 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090973/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090973/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=8f1c6124bafa46b09c274db8d4bab4b8 Command failed on smithi081 with status 2: 'cd /home/ubuntu/cephtest/s3-tests && ./bootstrap' [5090974] rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-comp-snappy.yaml overrides.yaml rgw_pool_type/ec.yaml tasks/rgw_roundtrip.yaml} ----------------------------------------------------------------- time: 00:08:18 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090974/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090974/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=b177b2ee7db6496ab8b1d5250dd3a473 Command failed on smithi012 with status 2: 'cd /home/ubuntu/cephtest/s3-tests && ./bootstrap' [5090975] rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-comp-zlib.yaml overrides.yaml proto/http.yaml rgw_pool_type/replicated.yaml striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:13:03 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090975/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090975/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=2617bffb8598421b968d4cbfd820b22e Command failed on smithi122 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5090977] rgw/verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-comp-zstd.yaml overrides.yaml proto/https.yaml rgw_pool_type/ec-profile.yaml striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests.yaml swift.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:13:55 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090977/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090977/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=3e597397f31c41119f170dd591e5c655 Command failed on smithi174 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5090978] rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-stupid.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml striping$/{stripe-equals-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:13:29 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090978/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090978/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=a91a51da57184bbc97c16f725d305d12 Command failed on smithi066 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5090979] rgw/hadoop-s3a/{clusters/fixed-2.yaml hadoop/v27.yaml s3a-hadoop.yaml supported-random-distro$/{ubuntu_latest.yaml}} ----------------------------------------------------------------- time: 00:11:48 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090979/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090979/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=5e21c9632ef94f21928f7a8ecc32b901 Command failed on smithi200 with status 1: 'cd /home/ubuntu/cephtest/hadoop/hadoop-tools/hadoop-aws/ && rm src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java && /home/ubuntu/cephtest/apache-maven-3.3.9/bin/mvn test -Dtest=S3a*,TestS3A*' [5090980] rgw/multisite/{clusters.yaml frontend/beast.yaml omap_limits.yaml overrides.yaml realms/three-zone-plus-pubsub.yaml tasks/test_multi.yaml valgrind.yaml} ----------------------------------------------------------------- time: 01:42:09 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090980/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090980/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=fc80cdb175f94196ae91cba8692ac473 rgw multisite, test failures [5090981] rgw/tempest/{clusters/fixed-1.yaml frontend/beast.yaml tasks/rgw_tempest.yaml ubuntu_latest.yaml} ----------------------------------------------------------------- time: 00:10:53 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090981/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090981/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=5177a4eef5284bd4ac1e2603f15d9e13 Command failed on smithi167 with status 1: "cd /home/ubuntu/cephtest/keystone && source .tox/venv/bin/activate && openstack project create --os-token ADMIN --os-url http://smithi167.front.sepia.ceph.com:35357/v2.0 --description 'Admin Tenant' admin" [5090982] rgw/verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/https.yaml rgw_pool_type/replicated.yaml striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests.yaml swift.yaml} validater/lockdep.yaml} ----------------------------------------------------------------- time: 00:13:42 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090982/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090982/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=46812b20c61d40089f18dc2836ec13bc Command failed on smithi040 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5090983] rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec-profile.yaml striping$/{stripe-greater-than-chunk.yaml} tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} ----------------------------------------------------------------- time: 00:13:02 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090983/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090983/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=a66c8c8fbd0343f8967bca493703d3e6 Command failed on smithi130 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5090984] rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/bluestore-comp-zstd.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_readwrite.yaml} ----------------------------------------------------------------- time: 00:12:16 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090984/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090984/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=54271b821f2f44f5bd98aab75a0d0cca Command failed on smithi096 with status 2: 'cd /home/ubuntu/cephtest/s3-tests && ./bootstrap' Pass ================================================================= [5090976] rgw/multifs/{clusters/fixed-2.yaml frontend/civetweb.yaml objectstore/bluestore-comp-zlib.yaml overrides.yaml rgw_pool_type/replicated.yaml tasks/rgw_s3tests.yaml} time: 00:43:14 info: http://pulpito.ceph.com/kchai-2020-05-25_05:27:30-rgw-nautilus-distro-basic-smithi/5090976/ 2020-05-25T07:40:34.081 INFO:teuthology.results:starting coverage generation 2020-05-25T07:40:34.087 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 35, in main int(args['--timeout']), args['--dry-run']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 84, in results generate_coverage(archive_dir, name) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 107, in generate_coverage archive_dir, File "/usr/lib/python3.6/subprocess.py", line 729, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage': '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage'