Found 367 jobs
Ignoring malparsed backtrace:  ceph version 16.0.0-8608-g431e6540 (431e6540d4b3bc60874aeba1c967cebe2bb78345) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x7f602175b826]
,  2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x557b9d779fe6]

Ignoring malparsed backtrace: 2020-12-30T15:59:55.850 INFO:teuthology.orchestra.run.smithi062.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards.
, 2020-12-30T15:59:56.395 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub
, noscrub is unset

Ignoring malparsed backtrace: 2020-12-30T16:00:23.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574956 got 55834574957 for osd.6
, 2020-12-30T16:00:23.384 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7
, 2020-12-30T16:00:23.648 INFO:teuthology.orchestra.run.smithi186.stdout:55834574955

Ignoring malparsed backtrace: 2020-12-30T16:00:30.355 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-12-30T16:00:02.615923+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=2, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2020-12-30T16:00:11.331651+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=11, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2020-12-30T15:59:55.655255+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=15, tm_min=59, tm_sec=55, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)

Ignoring malparsed backtrace: 2020-12-30T16:01:11.793 INFO:tasks.ceph.mgr.y:Stopped
, 2020-12-30T16:01:11.794 DEBUG:tasks.ceph.mgr.x:waiting for process to exit
, 2020-12-30T16:01:11.795 INFO:teuthology.orchestra.run:waiting for 300

Ignoring malparsed backtrace: teuthology.exceptions.CommandCrashedError: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 20 --max-in-flight 8 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 0 --op write 0 --op delete 20 --op append 5 --op write_excl 0 --op append_excl 5 --pool unique_pool_0'
, 2020-12-30T16:02:38.692 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits...
, 2020-12-30T16:02:38.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits

Ignoring malparsed backtrace:     fetch_binaries_for_coredumps(path, rem)
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/__init__.py", line 318, in fetch_binaries_for_coredumps
,     dump_program = dump_out.split("from '")[1].split(' ')[0]

Ignoring malparsed backtrace:  ceph version 16.0.0-8608-g431e6540 (431e6540d4b3bc60874aeba1c967cebe2bb78345) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x7f602175b826]
,  2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x557b9d779fe6]

Ignoring malparsed backtrace: 2020-12-30T15:59:55.850 INFO:teuthology.orchestra.run.smithi062.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards.
, 2020-12-30T15:59:56.395 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub
, noscrub is unset

Ignoring malparsed backtrace: 2020-12-30T16:00:23.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574956 got 55834574957 for osd.6
, 2020-12-30T16:00:23.384 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7
, 2020-12-30T16:00:23.648 INFO:teuthology.orchestra.run.smithi186.stdout:55834574955

Ignoring malparsed backtrace: 2020-12-30T16:00:30.355 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-12-30T16:00:02.615923+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=2, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2020-12-30T16:00:11.331651+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=11, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2020-12-30T15:59:55.655255+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=15, tm_min=59, tm_sec=55, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)

Ignoring malparsed backtrace: 2020-12-30T16:01:11.793 INFO:tasks.ceph.mgr.y:Stopped
, 2020-12-30T16:01:11.794 DEBUG:tasks.ceph.mgr.x:waiting for process to exit
, 2020-12-30T16:01:11.795 INFO:teuthology.orchestra.run:waiting for 300

Ignoring malparsed backtrace: teuthology.exceptions.CommandCrashedError: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 20 --max-in-flight 8 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 0 --op write 0 --op delete 20 --op append 5 --op write_excl 0 --op append_excl 5 --pool unique_pool_0'
, 2020-12-30T16:02:38.692 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits...
, 2020-12-30T16:02:38.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits

Ignoring malparsed backtrace:     fetch_binaries_for_coredumps(path, rem)
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/__init__.py", line 318, in fetch_binaries_for_coredumps
,     dump_program = dump_out.split("from '")[1].split(' ')[0]

Ignoring malparsed backtrace:  ceph version 16.0.0-8608-g431e6540 (431e6540d4b3bc60874aeba1c967cebe2bb78345) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x7f602175b826]
,  2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x557b9d779fe6]

Ignoring malparsed backtrace: 2020-12-30T15:59:55.850 INFO:teuthology.orchestra.run.smithi062.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards.
, 2020-12-30T15:59:56.395 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub
, noscrub is unset

Ignoring malparsed backtrace: 2020-12-30T16:00:23.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574956 got 55834574957 for osd.6
, 2020-12-30T16:00:23.384 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7
, 2020-12-30T16:00:23.648 INFO:teuthology.orchestra.run.smithi186.stdout:55834574955

Ignoring malparsed backtrace: 2020-12-30T16:00:30.355 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-12-30T16:00:02.615923+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=2, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2020-12-30T16:00:11.331651+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=11, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2020-12-30T15:59:55.655255+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=15, tm_min=59, tm_sec=55, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)

Ignoring malparsed backtrace: 2020-12-30T16:01:11.793 INFO:tasks.ceph.mgr.y:Stopped
, 2020-12-30T16:01:11.794 DEBUG:tasks.ceph.mgr.x:waiting for process to exit
, 2020-12-30T16:01:11.795 INFO:teuthology.orchestra.run:waiting for 300

Ignoring malparsed backtrace: teuthology.exceptions.CommandCrashedError: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 20 --max-in-flight 8 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 0 --op write 0 --op delete 20 --op append 5 --op write_excl 0 --op append_excl 5 --pool unique_pool_0'
, 2020-12-30T16:02:38.692 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits...
, 2020-12-30T16:02:38.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits

Ignoring malparsed backtrace:     fetch_binaries_for_coredumps(path, rem)
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/__init__.py", line 318, in fetch_binaries_for_coredumps
,     dump_program = dump_out.split("from '")[1].split(' ')[0]

Ignoring malparsed backtrace:  ceph version 16.0.0-8608-g431e6540 (431e6540d4b3bc60874aeba1c967cebe2bb78345) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x7f602175b826]
,  2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x557b9d779fe6]

Ignoring malparsed backtrace: 2020-12-30T15:59:55.850 INFO:teuthology.orchestra.run.smithi062.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards.
, 2020-12-30T15:59:56.395 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub
, noscrub is unset

Ignoring malparsed backtrace: 2020-12-30T16:00:23.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574956 got 55834574957 for osd.6
, 2020-12-30T16:00:23.384 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7
, 2020-12-30T16:00:23.648 INFO:teuthology.orchestra.run.smithi186.stdout:55834574955

Ignoring malparsed backtrace: 2020-12-30T16:00:30.355 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-12-30T16:00:02.615923+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=2, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2020-12-30T16:00:11.331651+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=11, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2020-12-30T15:59:55.655255+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=15, tm_min=59, tm_sec=55, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)

Ignoring malparsed backtrace: 2020-12-30T16:01:11.793 INFO:tasks.ceph.mgr.y:Stopped
, 2020-12-30T16:01:11.794 DEBUG:tasks.ceph.mgr.x:waiting for process to exit
, 2020-12-30T16:01:11.795 INFO:teuthology.orchestra.run:waiting for 300

Ignoring malparsed backtrace: teuthology.exceptions.CommandCrashedError: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 20 --max-in-flight 8 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 0 --op write 0 --op delete 20 --op append 5 --op write_excl 0 --op append_excl 5 --pool unique_pool_0'
, 2020-12-30T16:02:38.692 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits...
, 2020-12-30T16:02:38.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits

Ignoring malparsed backtrace:     fetch_binaries_for_coredumps(path, rem)
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/__init__.py", line 318, in fetch_binaries_for_coredumps
,     dump_program = dump_out.split("from '")[1].split(' ')[0]

Ignoring malparsed backtrace:  ceph version 16.0.0-8608-g431e6540 (431e6540d4b3bc60874aeba1c967cebe2bb78345) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x7f602175b826]
,  2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x557b9d779fe6]

Ignoring malparsed backtrace: 2020-12-30T15:59:55.850 INFO:teuthology.orchestra.run.smithi062.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards.
, 2020-12-30T15:59:56.395 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub
, noscrub is unset

Ignoring malparsed backtrace: 2020-12-30T16:00:23.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574956 got 55834574957 for osd.6
, 2020-12-30T16:00:23.384 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7
, 2020-12-30T16:00:23.648 INFO:teuthology.orchestra.run.smithi186.stdout:55834574955

Ignoring malparsed backtrace: 2020-12-30T16:00:30.355 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-12-30T16:00:02.615923+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=2, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2020-12-30T16:00:11.331651+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=11, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2020-12-30T15:59:55.655255+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=15, tm_min=59, tm_sec=55, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)

Ignoring malparsed backtrace: 2020-12-30T16:01:11.793 INFO:tasks.ceph.mgr.y:Stopped
, 2020-12-30T16:01:11.794 DEBUG:tasks.ceph.mgr.x:waiting for process to exit
, 2020-12-30T16:01:11.795 INFO:teuthology.orchestra.run:waiting for 300

Ignoring malparsed backtrace: teuthology.exceptions.CommandCrashedError: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 20 --max-in-flight 8 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 0 --op write 0 --op delete 20 --op append 5 --op write_excl 0 --op append_excl 5 --pool unique_pool_0'
, 2020-12-30T16:02:38.692 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits...
, 2020-12-30T16:02:38.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits

Ignoring malparsed backtrace:     fetch_binaries_for_coredumps(path, rem)
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/__init__.py", line 318, in fetch_binaries_for_coredumps
,     dump_program = dump_out.split("from '")[1].split(' ')[0]

Ignoring malparsed backtrace:  ceph version 16.0.0-8608-g431e6540 (431e6540d4b3bc60874aeba1c967cebe2bb78345) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x7f602175b826]
,  2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x557b9d779fe6]

Ignoring malparsed backtrace: 2020-12-30T15:59:55.850 INFO:teuthology.orchestra.run.smithi062.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards.
, 2020-12-30T15:59:56.395 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub
, noscrub is unset

Ignoring malparsed backtrace: 2020-12-30T16:00:23.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574956 got 55834574957 for osd.6
, 2020-12-30T16:00:23.384 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7
, 2020-12-30T16:00:23.648 INFO:teuthology.orchestra.run.smithi186.stdout:55834574955

Ignoring malparsed backtrace: 2020-12-30T16:00:30.355 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-12-30T16:00:02.615923+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=2, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2020-12-30T16:00:11.331651+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=11, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2020-12-30T15:59:55.655255+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=15, tm_min=59, tm_sec=55, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)

Ignoring malparsed backtrace: 2020-12-30T16:01:11.793 INFO:tasks.ceph.mgr.y:Stopped
, 2020-12-30T16:01:11.794 DEBUG:tasks.ceph.mgr.x:waiting for process to exit
, 2020-12-30T16:01:11.795 INFO:teuthology.orchestra.run:waiting for 300

Ignoring malparsed backtrace: teuthology.exceptions.CommandCrashedError: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 20 --max-in-flight 8 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 0 --op write 0 --op delete 20 --op append 5 --op write_excl 0 --op append_excl 5 --pool unique_pool_0'
, 2020-12-30T16:02:38.692 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits...
, 2020-12-30T16:02:38.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits

Ignoring malparsed backtrace:     fetch_binaries_for_coredumps(path, rem)
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/__init__.py", line 318, in fetch_binaries_for_coredumps
,     dump_program = dump_out.split("from '")[1].split(' ')[0]

Ignoring malparsed backtrace:  ceph version 16.0.0-8608-g431e6540 (431e6540d4b3bc60874aeba1c967cebe2bb78345) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x7f602175b826]
,  2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x666) [0x557b9d779fe6]

Ignoring malparsed backtrace: 2020-12-30T15:59:55.850 INFO:teuthology.orchestra.run.smithi062.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards.
, 2020-12-30T15:59:56.395 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub
, noscrub is unset

Ignoring malparsed backtrace: 2020-12-30T16:00:23.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574956 got 55834574957 for osd.6
, 2020-12-30T16:00:23.384 DEBUG:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7
, 2020-12-30T16:00:23.648 INFO:teuthology.orchestra.run.smithi186.stdout:55834574955

Ignoring malparsed backtrace: 2020-12-30T16:00:30.355 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2020-12-30T16:00:02.615923+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=2, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2020-12-30T16:00:11.331651+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=11, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)
, 2020-12-30T16:00:30.356 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2020-12-30T15:59:55.655255+0000 time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=15, tm_min=59, tm_sec=55, tm_wday=2, tm_yday=365, tm_isdst=-1) <= time.struct_time(tm_year=2020, tm_mon=12, tm_mday=30, tm_hour=16, tm_min=0, tm_sec=25, tm_wday=2, tm_yday=365, tm_isdst=0)

Ignoring malparsed backtrace: 2020-12-30T16:01:11.793 INFO:tasks.ceph.mgr.y:Stopped
, 2020-12-30T16:01:11.794 DEBUG:tasks.ceph.mgr.x:waiting for process to exit
, 2020-12-30T16:01:11.795 INFO:teuthology.orchestra.run:waiting for 300

Ignoring malparsed backtrace: teuthology.exceptions.CommandCrashedError: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 20 --max-in-flight 8 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 0 --op write 0 --op delete 20 --op append 5 --op write_excl 0 --op append_excl 5 --pool unique_pool_0'
, 2020-12-30T16:02:38.692 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits...
, 2020-12-30T16:02:38.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits

Ignoring malparsed backtrace:     fetch_binaries_for_coredumps(path, rem)
,   File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/__init__.py", line 318, in fetch_binaries_for_coredumps
,     dump_program = dump_out.split("from '")[1].split(' ')[0]

Ignoring malparsed backtrace:  ceph version 16.0.0-8608-g431e6540 (431e6540d4b3bc60874aeba1c967cebe2bb78345) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x7f8772a8a826]
,  2: (ReadOp::_finish(TestOp::CallbackInfo*)+0x1306) [0x55c3fb80c306]

Ignoring malparsed backtrace: 2020-12-30T16:20:57.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json
, 2020-12-30T16:20:57.519 INFO:teuthology.orchestra.run.smithi042.stdout:
, dumped all

Ignoring malparsed backtrace: 2020-12-30T16:22:19.610 INFO:teuthology.orchestra.run:waiting for 300
, 2020-12-30T16:22:19.699 INFO:tasks.ceph.osd.5:Stopped
, 2020-12-30T16:22:19.700 INFO:teuthology.misc:Shutting down mgr daemons...

Ignoring malparsed backtrace: 2020-12-30T16:27:09.289 INFO:teuthology.orchestra.run.smithi140.stdout:^+ hv03.front.sepia.ceph.com     3   6   377    61  -1169us[-1184us] +/-   88ms
, 2020-12-30T16:27:09.289 INFO:teuthology.orchestra.run.smithi140.stdout:^+ hv04.front.sepia.ceph.com     3   6   377    60  +1967us[+1967us] +/-   62ms
, 2020-12-30T16:27:09.290 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab

Ignoring malparsed backtrace:  ceph version 16.0.0-8608-g431e6540 (431e6540d4b3bc60874aeba1c967cebe2bb78345) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x7f8772a8a826]
,  2: (ReadOp::_finish(TestOp::CallbackInfo*)+0x1306) [0x55c3fb80c306]

Ignoring malparsed backtrace: 2020-12-30T16:20:57.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json
, 2020-12-30T16:20:57.519 INFO:teuthology.orchestra.run.smithi042.stdout:
, dumped all

Ignoring malparsed backtrace: 2020-12-30T16:22:19.610 INFO:teuthology.orchestra.run:waiting for 300
, 2020-12-30T16:22:19.699 INFO:tasks.ceph.osd.5:Stopped
, 2020-12-30T16:22:19.700 INFO:teuthology.misc:Shutting down mgr daemons...

Ignoring malparsed backtrace: 2020-12-30T16:27:09.289 INFO:teuthology.orchestra.run.smithi140.stdout:^+ hv03.front.sepia.ceph.com     3   6   377    61  -1169us[-1184us] +/-   88ms
, 2020-12-30T16:27:09.289 INFO:teuthology.orchestra.run.smithi140.stdout:^+ hv04.front.sepia.ceph.com     3   6   377    60  +1967us[+1967us] +/-   62ms
, 2020-12-30T16:27:09.290 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab

Ignoring malparsed backtrace:  ceph version 16.0.0-8608-g431e6540 (431e6540d4b3bc60874aeba1c967cebe2bb78345) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x7f8772a8a826]
,  2: (ReadOp::_finish(TestOp::CallbackInfo*)+0x1306) [0x55c3fb80c306]

Ignoring malparsed backtrace: 2020-12-30T16:20:57.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json
, 2020-12-30T16:20:57.519 INFO:teuthology.orchestra.run.smithi042.stdout:
, dumped all

Ignoring malparsed backtrace: 2020-12-30T16:22:19.610 INFO:teuthology.orchestra.run:waiting for 300
, 2020-12-30T16:22:19.699 INFO:tasks.ceph.osd.5:Stopped
, 2020-12-30T16:22:19.700 INFO:teuthology.misc:Shutting down mgr daemons...

Ignoring malparsed backtrace: 2020-12-30T16:27:09.289 INFO:teuthology.orchestra.run.smithi140.stdout:^+ hv03.front.sepia.ceph.com     3   6   377    61  -1169us[-1184us] +/-   88ms
, 2020-12-30T16:27:09.289 INFO:teuthology.orchestra.run.smithi140.stdout:^+ hv04.front.sepia.ceph.com     3   6   377    60  +1967us[+1967us] +/-   62ms
, 2020-12-30T16:27:09.290 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab

Ignoring malparsed backtrace:  ceph version 16.0.0-8608-g431e6540 (431e6540d4b3bc60874aeba1c967cebe2bb78345) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x7f8772a8a826]
,  2: (ReadOp::_finish(TestOp::CallbackInfo*)+0x1306) [0x55c3fb80c306]

Ignoring malparsed backtrace: 2020-12-30T16:20:57.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json
, 2020-12-30T16:20:57.519 INFO:teuthology.orchestra.run.smithi042.stdout:
, dumped all

Ignoring malparsed backtrace: 2020-12-30T16:22:19.610 INFO:teuthology.orchestra.run:waiting for 300
, 2020-12-30T16:22:19.699 INFO:tasks.ceph.osd.5:Stopped
, 2020-12-30T16:22:19.700 INFO:teuthology.misc:Shutting down mgr daemons...

Ignoring malparsed backtrace: 2020-12-30T16:27:09.289 INFO:teuthology.orchestra.run.smithi140.stdout:^+ hv03.front.sepia.ceph.com     3   6   377    61  -1169us[-1184us] +/-   88ms
, 2020-12-30T16:27:09.289 INFO:teuthology.orchestra.run.smithi140.stdout:^+ hv04.front.sepia.ceph.com     3   6   377    60  +1967us[+1967us] +/-   62ms
, 2020-12-30T16:27:09.290 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab

Ignoring malparsed backtrace:  ceph version 16.0.0-8608-g431e6540 (431e6540d4b3bc60874aeba1c967cebe2bb78345) pacific (dev)
,  1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe5) [0x7f8772a8a826]
,  2: (ReadOp::_finish(TestOp::CallbackInfo*)+0x1306) [0x55c3fb80c306]

Ignoring malparsed backtrace: 2020-12-30T16:20:57.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json
, 2020-12-30T16:20:57.519 INFO:teuthology.orchestra.run.smithi042.stdout:
, dumped all

Ignoring malparsed backtrace: 2020-12-30T16:22:19.610 INFO:teuthology.orchestra.run:waiting for 300
, 2020-12-30T16:22:19.699 INFO:tasks.ceph.osd.5:Stopped
, 2020-12-30T16:22:19.700 INFO:teuthology.misc:Shutting down mgr daemons...

Ignoring malparsed backtrace: 2020-12-30T16:27:09.289 INFO:teuthology.orchestra.run.smithi140.stdout:^+ hv03.front.sepia.ceph.com     3   6   377    61  -1169us[-1184us] +/-   88ms
, 2020-12-30T16:27:09.289 INFO:teuthology.orchestra.run.smithi140.stdout:^+ hv04.front.sepia.ceph.com     3   6   377    60  +1967us[+1967us] +/-   62ms
, 2020-12-30T16:27:09.290 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab

Found 11 distinct failure reasons
Timeout 6h running clone.client.0/qa/workunits/rados/test.sh
1 jobs: ['5747855']
suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v1only', 'objectstore/bluestore-comp-zstd', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_api_tests', 'validater/valgrind}']

Failure: reached maximum tries (180) after waiting for 180 seconds
3 jobs: ['5747938', '5748039', '5748161']
suites intersection: ['3-start-upgrade', '4-wait', 'fixed-2', 'rados/cephadm/upgrade/{1-start']
suites union: ['2-repo_digest/defaut', '2-repo_digest/repo_digest', '3-start-upgrade', '4-wait', 'distro$/{centos_8.0}', 'distro$/{rhel_8.0}', 'distro$/{rhel_latest}', 'fixed-2', 'mon_election/classic}', 'mon_election/connectivity}', 'rados/cephadm/upgrade/{1-start']

Dead: b'2020-12-31T04:17:43.122 DEBUG:teuthology.exit:Finished running handlers'
6 jobs: ['5748154', '5747960', '5747983', '5747894', '5748179', '5747926']
suites intersection: ['openstack}', 'rados', 'thrashosds-health']
suites union: ['1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-async-partial-recovery}', '2-recovery-overrides/{more-async-recovery}', 'backoff/normal', 'backoff/peering_and_degraded', 'ceph', 'cluster/{12-osds', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async', 'msgr/async-v1only', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/thrash-erasure-code-big/{ceph', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'recovery-overrides/{default}', 'recovery-overrides/{more-active-recovery}', 'recovery-overrides/{more-async-partial-recovery}', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/careful', 'thrashers/mapgap', 'thrashers/pggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}', 'workloads/radosbench-high-concurrency}', 'workloads/small-objects}']

Failure: Command failed on smithi077 with status 1: 'sudo docker kill -s 1 ceph-d16f0518-4aab-11eb-8f8d-001a4aab830c-osd.9'
1 jobs: ['5748101']
suites: ['1-install/nautilus-v1only', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'distro$/{ubuntu_18.04}', 'mon_election/classic', 'msgr-failures/osd-delay', 'rados', 'rados/cephadm/thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size', 'thrashers/mapgap', 'thrashosds-health', 'three-plus-one}', 'workloads/snaps-few-objects}']

Timeout 3h running clone.client.0/qa/workunits/rados/test.sh
1 jobs: ['5747936']
suites: ['clusters/9-mons', 'mon_election/connectivity', 'msgr-failures/mon-delay', 'msgr/async-v1only', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/monthrash/{ceph', 'supported-random-distro$/{centos_8}', 'thrashers/sync-many', 'workloads/rados_api_tests}']

Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi146 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=431e6540d4b3bc60874aeba1c967cebe2bb78345 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh'
2 jobs: ['5747878', '5748010']
suites intersection: ['rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}']
suites union: ['mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_cephadm}']

Failure: Test failure: test_delete (tasks.mgr.dashboard.test_rgw.RgwUserCapabilityTest)
1 jobs: ['5747993']
suites: ['debug/mgr', 'mon_election/classic', 'objectstore/filestore-xfs', 'rados/dashboard/{clusters/{2-node-mgr}', 'supported-random-distro$/{rhel_8}', 'tasks/dashboard}']

Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400000 --objects 20 --max-in-flight 8 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 0 --op write 0 --op delete 20 --op append 5 --op write_excl 0 --op append_excl 5 --pool unique_pool_0'
2 jobs: ['5748117', '5748135']
suites intersection: ['mon_election/classic', 'rados']
suites union: ['clusters/{fixed-2', 'fast/fast', 'mon_election/classic', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async-v1only', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zlib', 'openstack}', 'rados', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'rados/thrash-erasure-code/{ceph', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{rhel_8}', 'thrashers/fastread', 'thrashosds-health', 'workloads/ec-small-objects-many-deletes}']

Failure: timeout expired in wait_until_healthy
1 jobs: ['5748017']
suites: ['1-install/nautilus', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{rbd-cls', '5-workload/{radosbench', '6-finish-upgrade', '7-octopus', '8-workload/{rbd-python', 'bluestore-bitmap', 'mon_election/connectivity', 'rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack', 'rbd-import-export', 'rbd_api}', 'readwrite', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_latest}']

Dead: b"TypeError: argument of type 'NoneType' is not iterable"
1 jobs: ['5747913']
suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-async-partial-recovery}', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/osd-delay', 'msgr/async', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{rhel_8}', 'thrashers/mapgap', 'thrashosds-health', 'workloads/redirect}']

Failure: Command failed (workunit test cephadm/test_adoption.sh) on smithi047 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=431e6540d4b3bc60874aeba1c967cebe2bb78345 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_adoption.sh'
1 jobs: ['5748112']
suites: ['mon_election/classic', 'rados/cephadm/workunits/{distro/ubuntu_18.04_podman', 'task/test_adoption}']