2021-02-06T17:34:45.986 INFO:root:teuthology version: 1.1.0-17c09965 2021-02-06T17:34:45.987 INFO:teuthology.results:subset: '111/120000' 2021-02-06T17:34:45.988 INFO:teuthology.results:seed: '6568' 2021-02-06T20:54:27.712 INFO:root:teuthology version: 1.1.0-17c09965 2021-02-06T20:54:27.785 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-02-07T08:31:52.275 INFO:teuthology.results:Tests finished! gathering results... 2021-02-07T08:33:15.004 INFO:scrape:Found 361 jobs 2021-02-07T08:33:15.364 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-173-gb9c709c0 (b9c709c051a31b3730916788d9bcd37fcd4380c0) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fb6a2eb721a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x5570ad23df36] 2021-02-07T08:33:15.366 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-02-06T20:44:23.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-02-06T20:44:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: 2021-02-07T08:33:15.367 WARNING:scrape:Ignoring malparsed backtrace: 2021-02-06T20:45:28.926 INFO:tasks.ceph:Checking cluster log for badness... , 2021-02-06T20:45:28.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 , 2021-02-06T20:45:28.993 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2021-02-07T08:33:15.367 WARNING:scrape:Ignoring malparsed backtrace: 2021-02-06T20:47:51.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-02-06T20:47:51.311 DEBUG:teuthology.run_tasks:Unwinding manager pcp , 2021-02-06T20:47:51.323 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2021-02-07T08:33:15.474 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-173-gb9c709c0 (b9c709c051a31b3730916788d9bcd37fcd4380c0) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fb6a2eb721a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x5570ad23df36] 2021-02-07T08:33:15.476 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-02-06T20:44:23.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-02-06T20:44:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: 2021-02-07T08:33:15.476 WARNING:scrape:Ignoring malparsed backtrace: 2021-02-06T20:45:28.926 INFO:tasks.ceph:Checking cluster log for badness... , 2021-02-06T20:45:28.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 , 2021-02-06T20:45:28.993 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2021-02-07T08:33:15.477 WARNING:scrape:Ignoring malparsed backtrace: 2021-02-06T20:47:51.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-02-06T20:47:51.311 DEBUG:teuthology.run_tasks:Unwinding manager pcp , 2021-02-06T20:47:51.323 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2021-02-07T08:33:15.579 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-173-gb9c709c0 (b9c709c051a31b3730916788d9bcd37fcd4380c0) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fb6a2eb721a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x5570ad23df36] 2021-02-07T08:33:15.581 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-02-06T20:44:23.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-02-06T20:44:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: 2021-02-07T08:33:15.582 WARNING:scrape:Ignoring malparsed backtrace: 2021-02-06T20:45:28.926 INFO:tasks.ceph:Checking cluster log for badness... , 2021-02-06T20:45:28.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 , 2021-02-06T20:45:28.993 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2021-02-07T08:33:15.582 WARNING:scrape:Ignoring malparsed backtrace: 2021-02-06T20:47:51.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-02-06T20:47:51.311 DEBUG:teuthology.run_tasks:Unwinding manager pcp , 2021-02-06T20:47:51.323 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2021-02-07T08:33:15.684 WARNING:scrape:Ignoring malparsed backtrace: ceph version 16.1.0-173-gb9c709c0 (b9c709c051a31b3730916788d9bcd37fcd4380c0) pacific (rc) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe5) [0x7fb6a2eb721a] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x5b6) [0x5570ad23df36] 2021-02-07T08:33:15.686 WARNING:scrape:Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2021-02-06T20:44:23.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json , 2021-02-06T20:44:24.245 INFO:teuthology.orchestra.run.smithi007.stdout: 2021-02-07T08:33:15.686 WARNING:scrape:Ignoring malparsed backtrace: 2021-02-06T20:45:28.926 INFO:tasks.ceph:Checking cluster log for badness... , 2021-02-06T20:45:28.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 , 2021-02-06T20:45:28.993 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2021-02-07T08:33:15.687 WARNING:scrape:Ignoring malparsed backtrace: 2021-02-06T20:47:51.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 , 2021-02-06T20:47:51.311 DEBUG:teuthology.run_tasks:Unwinding manager pcp , 2021-02-06T20:47:51.323 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2021-02-07T08:33:15.777 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2021-02-06_17:33:15-rados-wip-yuri-testing-2021-02-04-1045-pacific-distro-basic-smithi/5862164/teuthology.log 2021-02-07T08:33:15.814 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/results.py", line 34, in main int(args['--timeout']), args['--dry-run']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/results.py", line 69, in results Scraper(archive_dir).analyze() File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/scrape.py", line 490, in analyze if reason.match(job): File "/home/teuthworker/src/git.ceph.com_git_teuthology_17c099650191e7aa47a7d9f502facca74f9b4d44/teuthology/scrape.py", line 204, in match job.get_last_tlog_line()).ratio() File "/usr/lib/python3.6/difflib.py", line 644, in ratio matches = sum(triple[-1] for triple in self.get_matching_blocks()) File "/usr/lib/python3.6/difflib.py", line 467, in get_matching_blocks la, lb = len(self.a), len(self.b) TypeError: object of type 'NoneType' has no len()