Found 298 jobs Ignoring malparsed backtrace: ceph version 17.1.0-90-gf8bae789 (f8bae78956976eb41022af2d864d805106c0d6fd) quincy (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7fdb8452af17] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x652) [0x55e96300bdb2] Ignoring malparsed backtrace: 2022-03-25T22:41:51.105 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2022-03-25T22:41:51.105 INFO:teuthology.orchestra.run.smithi038.stdout:} , 2022-03-25T22:41:51.117 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 Ignoring malparsed backtrace: 2022-03-25T22:42:33.664 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 , 2022-03-25T22:42:33.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi038.front.sepia.ceph.com , 2022-03-25T22:42:33.954 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 Ignoring malparsed backtrace: 2022-03-25T22:42:51.567 INFO:teuthology.task.internal.syslog:Compressing syslogs... , 2022-03-25T22:42:51.568 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2022-03-25T22:42:51.595 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- Ignoring malparsed backtrace: ceph version 17.1.0-90-gf8bae789 (f8bae78956976eb41022af2d864d805106c0d6fd) quincy (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7fdb8452af17] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x652) [0x55e96300bdb2] Ignoring malparsed backtrace: 2022-03-25T22:41:51.105 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2022-03-25T22:41:51.105 INFO:teuthology.orchestra.run.smithi038.stdout:} , 2022-03-25T22:41:51.117 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 Ignoring malparsed backtrace: 2022-03-25T22:42:33.664 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 , 2022-03-25T22:42:33.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi038.front.sepia.ceph.com , 2022-03-25T22:42:33.954 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 Ignoring malparsed backtrace: 2022-03-25T22:42:51.567 INFO:teuthology.task.internal.syslog:Compressing syslogs... , 2022-03-25T22:42:51.568 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2022-03-25T22:42:51.595 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- Ignoring malparsed backtrace: ceph version 17.1.0-90-gf8bae789 (f8bae78956976eb41022af2d864d805106c0d6fd) quincy (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7fdb8452af17] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x652) [0x55e96300bdb2] Ignoring malparsed backtrace: 2022-03-25T22:41:51.105 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2022-03-25T22:41:51.105 INFO:teuthology.orchestra.run.smithi038.stdout:} , 2022-03-25T22:41:51.117 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 Ignoring malparsed backtrace: 2022-03-25T22:42:33.664 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 , 2022-03-25T22:42:33.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi038.front.sepia.ceph.com , 2022-03-25T22:42:33.954 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 Ignoring malparsed backtrace: 2022-03-25T22:42:51.567 INFO:teuthology.task.internal.syslog:Compressing syslogs... , 2022-03-25T22:42:51.568 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2022-03-25T22:42:51.595 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- Ignoring malparsed backtrace: ceph version 17.1.0-90-gf8bae789 (f8bae78956976eb41022af2d864d805106c0d6fd) quincy (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7fdb8452af17] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x652) [0x55e96300bdb2] Ignoring malparsed backtrace: 2022-03-25T22:41:51.105 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2022-03-25T22:41:51.105 INFO:teuthology.orchestra.run.smithi038.stdout:} , 2022-03-25T22:41:51.117 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 Ignoring malparsed backtrace: 2022-03-25T22:42:33.664 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 , 2022-03-25T22:42:33.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi038.front.sepia.ceph.com , 2022-03-25T22:42:33.954 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 Ignoring malparsed backtrace: 2022-03-25T22:42:51.567 INFO:teuthology.task.internal.syslog:Compressing syslogs... , 2022-03-25T22:42:51.568 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2022-03-25T22:42:51.595 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- Ignoring malparsed backtrace: ceph version 17.1.0-90-gf8bae789 (f8bae78956976eb41022af2d864d805106c0d6fd) quincy (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7fdb8452af17] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x652) [0x55e96300bdb2] Ignoring malparsed backtrace: 2022-03-25T22:41:51.105 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2022-03-25T22:41:51.105 INFO:teuthology.orchestra.run.smithi038.stdout:} , 2022-03-25T22:41:51.117 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 Ignoring malparsed backtrace: 2022-03-25T22:42:33.664 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 , 2022-03-25T22:42:33.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi038.front.sepia.ceph.com , 2022-03-25T22:42:33.954 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 Ignoring malparsed backtrace: 2022-03-25T22:42:51.567 INFO:teuthology.task.internal.syslog:Compressing syslogs... , 2022-03-25T22:42:51.568 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2022-03-25T22:42:51.595 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- Ignoring malparsed backtrace: ceph version 17.1.0-90-gf8bae789 (f8bae78956976eb41022af2d864d805106c0d6fd) quincy (dev) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xe0) [0x7fdb8452af17] , 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x652) [0x55e96300bdb2] Ignoring malparsed backtrace: 2022-03-25T22:41:51.105 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2022-03-25T22:41:51.105 INFO:teuthology.orchestra.run.smithi038.stdout:} , 2022-03-25T22:41:51.117 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 Ignoring malparsed backtrace: 2022-03-25T22:42:33.664 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 , 2022-03-25T22:42:33.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi038.front.sepia.ceph.com , 2022-03-25T22:42:33.954 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 Ignoring malparsed backtrace: 2022-03-25T22:42:51.567 INFO:teuthology.task.internal.syslog:Compressing syslogs... , 2022-03-25T22:42:51.568 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- , 2022-03-25T22:42:51.595 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- Found 4 distinct failure reasons Failure: Test failure: test_export_create_with_non_existing_fsname (tasks.cephfs.test_nfs.TestNFS) 1 jobs: ['6761976'] suites: ['mon_election/classic', 'rados/cephadm/workunits/{agent/on', 'task/test_nfs}'] Failure: Command failed on smithi174 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f8bae78956976eb41022af2d864d805106c0d6fd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a098e73e-ac88-11ec-8c35-001a4aab830c -- ceph mon dump -f json' 1 jobs: ['6762077'] suites: ['0-nvme-loop', '1-start', '2-ops/rm-zap-add}', 'rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0'] Failure: Command failed (workunit test rados/test.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f8bae78956976eb41022af2d864d805106c0d6fd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh' 2 jobs: ['6761982', '6761856'] suites intersection: ['rados', 'supported-random-distro$/{centos_8}'] suites union: ['clusters/3-mons', 'clusters/{fixed-2', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/many', 'msgr/async', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/monthrash/{ceph', 'supported-random-distro$/{centos_8}', 'tasks/rados_api_tests}', 'thrashers/sync', 'workloads/rados_api_tests}'] Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' 1 jobs: ['6762122'] suites: ['mon_election/classic', 'msgr-failures/none', 'msgr/async', 'objectstore/bluestore-comp-snappy', 'rados', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'supported-random-distro$/{ubuntu_latest}}']