2021-03-07T11:47:48.838 INFO:root:teuthology version: 1.1.0-3082387b
2021-03-07T11:47:48.841 INFO:teuthology.results:seed: '2477'
2021-03-07T11:48:33.919 INFO:root:teuthology version: 1.1.0-3082387b
2021-03-07T11:48:33.924 INFO:teuthology.results:Waiting up to 7200 seconds for tests to finish...
2021-03-07T12:06:35.257 INFO:teuthology.results:Tests finished! gathering results...
2021-03-07T12:06:35.430 INFO:scrape:Found 1 jobs
2021-03-07T12:06:35.446 WARNING:scrape:Ignoring malparsed backtrace:  ceph version 17.0.0-1644-gfba282a2 (fba282a258c58980ee5279c7900ae54397688776) quincy (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe0) [0x7f8af70f3638]
,  2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x6e2) [0x558817a62552]

2021-03-07T12:06:35.448 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-07T12:05:06.850 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0
, 2021-03-07T12:05:07.151 INFO:teuthology.orchestra.run.smithi098.stdout:34359738395
, 2021-03-07T12:05:07.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738401 got 34359738395 for osd.0

2021-03-07T12:06:35.448 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-07T12:05:35.417 ERROR:teuthology.contextutil:Saw exception from nested tasks
, Traceback (most recent call last):
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/contextutil.py", line 33, in nested

2021-03-07T12:06:35.449 WARNING:scrape:Ignoring malparsed backtrace:   File "src/gevent/greenlet.py", line 829, in gevent._gevent_cgreenlet.Greenlet.get
,   File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise

2021-03-07T12:06:35.449 WARNING:scrape:Ignoring malparsed backtrace:   File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/run.py", line 161, in wait
,     self._raise_for_status()
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/run.py", line 179, in _raise_for_status

2021-03-07T12:06:35.469 WARNING:scrape:Ignoring malparsed backtrace:  ceph version 17.0.0-1644-gfba282a2 (fba282a258c58980ee5279c7900ae54397688776) quincy (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe0) [0x7f8af70f3638]
,  2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x6e2) [0x558817a62552]

2021-03-07T12:06:35.471 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-07T12:05:06.850 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0
, 2021-03-07T12:05:07.151 INFO:teuthology.orchestra.run.smithi098.stdout:34359738395
, 2021-03-07T12:05:07.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738401 got 34359738395 for osd.0

2021-03-07T12:06:35.472 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-07T12:05:35.417 ERROR:teuthology.contextutil:Saw exception from nested tasks
, Traceback (most recent call last):
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/contextutil.py", line 33, in nested

2021-03-07T12:06:35.472 WARNING:scrape:Ignoring malparsed backtrace:   File "src/gevent/greenlet.py", line 829, in gevent._gevent_cgreenlet.Greenlet.get
,   File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise

2021-03-07T12:06:35.473 WARNING:scrape:Ignoring malparsed backtrace:   File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/run.py", line 161, in wait
,     self._raise_for_status()
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/run.py", line 179, in _raise_for_status

2021-03-07T12:06:35.493 WARNING:scrape:Ignoring malparsed backtrace:  ceph version 17.0.0-1644-gfba282a2 (fba282a258c58980ee5279c7900ae54397688776) quincy (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe0) [0x7f8af70f3638]
,  2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x6e2) [0x558817a62552]

2021-03-07T12:06:35.495 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-07T12:05:06.850 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0
, 2021-03-07T12:05:07.151 INFO:teuthology.orchestra.run.smithi098.stdout:34359738395
, 2021-03-07T12:05:07.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738401 got 34359738395 for osd.0

2021-03-07T12:06:35.496 WARNING:scrape:Ignoring malparsed backtrace: 2021-03-07T12:05:35.417 ERROR:teuthology.contextutil:Saw exception from nested tasks
, Traceback (most recent call last):
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/contextutil.py", line 33, in nested

2021-03-07T12:06:35.496 WARNING:scrape:Ignoring malparsed backtrace:   File "src/gevent/greenlet.py", line 829, in gevent._gevent_cgreenlet.Greenlet.get
,   File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise

2021-03-07T12:06:35.497 WARNING:scrape:Ignoring malparsed backtrace:   File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/run.py", line 161, in wait
,     self._raise_for_status()
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/run.py", line 179, in _raise_for_status

2021-03-07T12:06:35.503 INFO:scrape:Found 1 distinct failure reasons
2021-03-07T12:06:35.504 INFO:scrape:Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 10 --op write_excl 50 --pool unique_pool_0'
2021-03-07T12:06:35.504 INFO:scrape:1 jobs: ['5944366']
2021-03-07T12:06:35.505 INFO:scrape:suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'centos_latest', 'clusters/{fixed-2}', 'crimson-rados:thrash/{0-size-min-size-overrides/3-size-2-min-size', 'deploy/ceph', 'objectstore/bluestore', 'thrashers/default', 'thrashosds-health', 'workloads/small-objects}']
2021-03-07T12:06:35.505 INFO:scrape:
2021-03-07T12:06:35.506 INFO:teuthology.results:Sending results to xxhdx1985126@gmail.com: Test Run: xuxuehan-2021-03-07_11:46:08-crimson-rados:thrash-wip-crimson-interruptible-future-pipeline-parallelism-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/xuxuehan-2021-03-07_11:46:08-crimson-rados:thrash-wip-crimson-interruptible-future-pipeline-parallelism-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/xuxuehan-2021-03-07_11:46:08-crimson-rados:thrash-wip-crimson-interruptible-future-pipeline-parallelism-distro-basic-smithi/
failed:  1
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[5944366]  crimson-rados:thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects}
-----------------------------------------------------------------
time:   00:08:22
info:   http://pulpito.ceph.com/xuxuehan-2021-03-07_11:46:08-crimson-rados:thrash-wip-crimson-interruptible-future-pipeline-parallelism-distro-basic-smithi/5944366/
log:    http://qa-proxy.ceph.com/teuthology/xuxuehan-2021-03-07_11:46:08-crimson-rados:thrash-wip-crimson-interruptible-future-pipeline-parallelism-distro-basic-smithi/5944366/

    Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage
    /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000
    --objects 1024 --max-in-flight 64 --size 4000000 --min-stride-size 400000
    --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op
    delete 10 --op write_excl 50 --pool unique_pool_0'