Found 1 jobs Ignoring malparsed backtrace: ceph version 17.0.0-1647-gd801ec56 (d801ec56fd883d1df41f872b9211c1534fb4ac9c) quincy (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7fee91fa0638] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x6e2) [0x557a8eb9b552] Ignoring malparsed backtrace: 2021-03-08T12:52:30.665 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, , 2021-03-08T12:52:30.666 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", , 2021-03-08T12:52:30.666 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, Ignoring malparsed backtrace: 2021-03-08T12:52:30.695 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, , 2021-03-08T12:52:30.695 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], , 2021-03-08T12:52:30.695 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, Ignoring malparsed backtrace: 2021-03-08T12:52:30.720 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, , 2021-03-08T12:52:30.720 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { , 2021-03-08T12:52:30.720 INFO:teuthology.orchestra.run.smithi047.stdout: "rados": {} Ignoring malparsed backtrace: 2021-03-08T12:52:30.744 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 28559 , 2021-03-08T12:52:30.744 INFO:teuthology.orchestra.run.smithi047.stdout: } , 2021-03-08T12:52:30.744 INFO:teuthology.orchestra.run.smithi047.stdout: ] Ignoring malparsed backtrace: 2021-03-08T12:52:30.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, , 2021-03-08T12:52:30.767 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { , 2021-03-08T12:52:30.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ Ignoring malparsed backtrace: 2021-03-08T12:52:30.788 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode_enabled": false, , 2021-03-08T12:52:30.789 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_bucket_count": 0, , 2021-03-08T12:52:30.789 INFO:teuthology.orchestra.run.smithi047.stdout: "degraded_stretch_mode": 0, Ignoring malparsed backtrace: instructed osd(s) 1 to deep-scrub , 2021-03-08T12:52:52.086 INFO:tasks.ceph:Scrubbing osd.2 , 2021-03-08T12:52:52.087 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 Ignoring malparsed backtrace: 2021-03-08T12:53:14.419 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 , 2021-03-08T12:53:14.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi047.front.sepia.ceph.com , 2021-03-08T12:53:14.585 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 Ignoring malparsed backtrace: 2021-03-08T12:53:16.307 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart , Redirecting to /bin/systemctl restart rsyslog.service , Redirecting to /bin/systemctl restart rsyslog.service Ignoring malparsed backtrace: ceph version 17.0.0-1647-gd801ec56 (d801ec56fd883d1df41f872b9211c1534fb4ac9c) quincy (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7fee91fa0638] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x6e2) [0x557a8eb9b552] Ignoring malparsed backtrace: 2021-03-08T12:52:30.665 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, , 2021-03-08T12:52:30.666 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", , 2021-03-08T12:52:30.666 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, Ignoring malparsed backtrace: 2021-03-08T12:52:30.695 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, , 2021-03-08T12:52:30.695 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], , 2021-03-08T12:52:30.695 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, Ignoring malparsed backtrace: 2021-03-08T12:52:30.720 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, , 2021-03-08T12:52:30.720 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { , 2021-03-08T12:52:30.720 INFO:teuthology.orchestra.run.smithi047.stdout: "rados": {} Ignoring malparsed backtrace: 2021-03-08T12:52:30.744 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 28559 , 2021-03-08T12:52:30.744 INFO:teuthology.orchestra.run.smithi047.stdout: } , 2021-03-08T12:52:30.744 INFO:teuthology.orchestra.run.smithi047.stdout: ] Ignoring malparsed backtrace: 2021-03-08T12:52:30.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, , 2021-03-08T12:52:30.767 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { , 2021-03-08T12:52:30.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ Ignoring malparsed backtrace: 2021-03-08T12:52:30.788 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode_enabled": false, , 2021-03-08T12:52:30.789 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_bucket_count": 0, , 2021-03-08T12:52:30.789 INFO:teuthology.orchestra.run.smithi047.stdout: "degraded_stretch_mode": 0, Ignoring malparsed backtrace: instructed osd(s) 1 to deep-scrub , 2021-03-08T12:52:52.086 INFO:tasks.ceph:Scrubbing osd.2 , 2021-03-08T12:52:52.087 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 Ignoring malparsed backtrace: 2021-03-08T12:53:14.419 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 , 2021-03-08T12:53:14.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi047.front.sepia.ceph.com , 2021-03-08T12:53:14.585 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 Ignoring malparsed backtrace: 2021-03-08T12:53:16.307 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart , Redirecting to /bin/systemctl restart rsyslog.service , Redirecting to /bin/systemctl restart rsyslog.service Ignoring malparsed backtrace: ceph version 17.0.0-1647-gd801ec56 (d801ec56fd883d1df41f872b9211c1534fb4ac9c) quincy (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7fee91fa0638] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x6e2) [0x557a8eb9b552] Ignoring malparsed backtrace: 2021-03-08T12:52:30.665 INFO:teuthology.orchestra.run.smithi047.stdout: "object_hash": 2, , 2021-03-08T12:52:30.666 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscale_mode": "off", , 2021-03-08T12:52:30.666 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": 1, Ignoring malparsed backtrace: 2021-03-08T12:52:30.695 INFO:teuthology.orchestra.run.smithi047.stdout: "quota_max_objects": 0, , 2021-03-08T12:52:30.695 INFO:teuthology.orchestra.run.smithi047.stdout: "tiers": [], , 2021-03-08T12:52:30.695 INFO:teuthology.orchestra.run.smithi047.stdout: "tier_of": -1, Ignoring malparsed backtrace: 2021-03-08T12:52:30.720 INFO:teuthology.orchestra.run.smithi047.stdout: "options": {}, , 2021-03-08T12:52:30.720 INFO:teuthology.orchestra.run.smithi047.stdout: "application_metadata": { , 2021-03-08T12:52:30.720 INFO:teuthology.orchestra.run.smithi047.stdout: "rados": {} Ignoring malparsed backtrace: 2021-03-08T12:52:30.744 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 28559 , 2021-03-08T12:52:30.744 INFO:teuthology.orchestra.run.smithi047.stdout: } , 2021-03-08T12:52:30.744 INFO:teuthology.orchestra.run.smithi047.stdout: ] Ignoring malparsed backtrace: 2021-03-08T12:52:30.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, , 2021-03-08T12:52:30.767 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_addrs": { , 2021-03-08T12:52:30.767 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ Ignoring malparsed backtrace: 2021-03-08T12:52:30.788 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_mode_enabled": false, , 2021-03-08T12:52:30.789 INFO:teuthology.orchestra.run.smithi047.stdout: "stretch_bucket_count": 0, , 2021-03-08T12:52:30.789 INFO:teuthology.orchestra.run.smithi047.stdout: "degraded_stretch_mode": 0, Ignoring malparsed backtrace: instructed osd(s) 1 to deep-scrub , 2021-03-08T12:52:52.086 INFO:tasks.ceph:Scrubbing osd.2 , 2021-03-08T12:52:52.087 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 Ignoring malparsed backtrace: 2021-03-08T12:53:14.419 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 , 2021-03-08T12:53:14.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi047.front.sepia.ceph.com , 2021-03-08T12:53:14.585 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 Ignoring malparsed backtrace: 2021-03-08T12:53:16.307 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart , Redirecting to /bin/systemctl restart rsyslog.service , Redirecting to /bin/systemctl restart rsyslog.service Found 1 distinct failure reasons Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 1024 --max-in-flight 64 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 10 --op write_excl 50 --pool unique_pool_0' 1 jobs: ['5946458'] suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{default}', 'centos_latest', 'clusters/{fixed-2}', 'crimson-rados:thrash/{0-size-min-size-overrides/3-size-2-min-size', 'deploy/ceph', 'objectstore/bluestore', 'thrashers/default', 'thrashosds-health', 'workloads/small-objects}']