Found 11 jobs Ignoring malparsed backtrace: ceph version 17.2.3-766-gf860b69d (f860b69d9ecbf6a250d17b83c1778c894700357d) quincy (stable) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x7f6a3c691a99] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x484) [0x55ed07b3cf74] Ignoring malparsed backtrace: 2022-09-13T14:32:43.531 INFO:teuthology.orchestra.run.smithi172.stdout:{ , 2022-09-13T14:32:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2022-09-13T14:32:43.532 INFO:teuthology.orchestra.run.smithi172.stdout:} Ignoring malparsed backtrace: 2022-09-13T14:33:46.486 INFO:tasks.ceph.osd.5:Stopped , 2022-09-13T14:33:46.488 INFO:teuthology.misc:Shutting down mgr daemons... , 2022-09-13T14:33:46.489 DEBUG:tasks.ceph.mgr.x:waiting for process to exit Ignoring malparsed backtrace: 2022-09-13T14:34:53.335 INFO:teuthology.task.ansible:Skipping ansible cleanup... , 2022-09-13T14:34:53.336 DEBUG:teuthology.run_tasks:Unwinding manager selinux , 2022-09-13T14:34:53.380 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/teuthology/orchestra/remote.py", line 96, in sh , proc = self.run(**kwargs) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/teuthology/orchestra/remote.py", line 510, in run Ignoring malparsed backtrace: ceph version 17.2.3-766-gf860b69d (f860b69d9ecbf6a250d17b83c1778c894700357d) quincy (stable) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x7f6a3c691a99] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x484) [0x55ed07b3cf74] Ignoring malparsed backtrace: 2022-09-13T14:32:43.531 INFO:teuthology.orchestra.run.smithi172.stdout:{ , 2022-09-13T14:32:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2022-09-13T14:32:43.532 INFO:teuthology.orchestra.run.smithi172.stdout:} Ignoring malparsed backtrace: 2022-09-13T14:33:46.486 INFO:tasks.ceph.osd.5:Stopped , 2022-09-13T14:33:46.488 INFO:teuthology.misc:Shutting down mgr daemons... , 2022-09-13T14:33:46.489 DEBUG:tasks.ceph.mgr.x:waiting for process to exit Ignoring malparsed backtrace: 2022-09-13T14:34:53.335 INFO:teuthology.task.ansible:Skipping ansible cleanup... , 2022-09-13T14:34:53.336 DEBUG:teuthology.run_tasks:Unwinding manager selinux , 2022-09-13T14:34:53.380 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/teuthology/orchestra/remote.py", line 96, in sh , proc = self.run(**kwargs) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/teuthology/orchestra/remote.py", line 510, in run Ignoring malparsed backtrace: ceph version 17.2.3-766-gf860b69d (f860b69d9ecbf6a250d17b83c1778c894700357d) quincy (stable) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x7f6a3c691a99] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x484) [0x55ed07b3cf74] Ignoring malparsed backtrace: 2022-09-13T14:32:43.531 INFO:teuthology.orchestra.run.smithi172.stdout:{ , 2022-09-13T14:32:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2022-09-13T14:32:43.532 INFO:teuthology.orchestra.run.smithi172.stdout:} Ignoring malparsed backtrace: 2022-09-13T14:33:46.486 INFO:tasks.ceph.osd.5:Stopped , 2022-09-13T14:33:46.488 INFO:teuthology.misc:Shutting down mgr daemons... , 2022-09-13T14:33:46.489 DEBUG:tasks.ceph.mgr.x:waiting for process to exit Ignoring malparsed backtrace: 2022-09-13T14:34:53.335 INFO:teuthology.task.ansible:Skipping ansible cleanup... , 2022-09-13T14:34:53.336 DEBUG:teuthology.run_tasks:Unwinding manager selinux , 2022-09-13T14:34:53.380 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/teuthology/orchestra/remote.py", line 96, in sh , proc = self.run(**kwargs) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/teuthology/orchestra/remote.py", line 510, in run Ignoring malparsed backtrace: ceph version 17.2.3-766-gf860b69d (f860b69d9ecbf6a250d17b83c1778c894700357d) quincy (stable) , 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x7f6a3c691a99] , 2: (CopyFromOp::_finish(TestOp::CallbackInfo*)+0x484) [0x55ed07b3cf74] Ignoring malparsed backtrace: 2022-09-13T14:32:43.531 INFO:teuthology.orchestra.run.smithi172.stdout:{ , 2022-09-13T14:32:43.532 INFO:teuthology.orchestra.run.smithi172.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " , 2022-09-13T14:32:43.532 INFO:teuthology.orchestra.run.smithi172.stdout:} Ignoring malparsed backtrace: 2022-09-13T14:33:46.486 INFO:tasks.ceph.osd.5:Stopped , 2022-09-13T14:33:46.488 INFO:teuthology.misc:Shutting down mgr daemons... , 2022-09-13T14:33:46.489 DEBUG:tasks.ceph.mgr.x:waiting for process to exit Ignoring malparsed backtrace: 2022-09-13T14:34:53.335 INFO:teuthology.task.ansible:Skipping ansible cleanup... , 2022-09-13T14:34:53.336 DEBUG:teuthology.run_tasks:Unwinding manager selinux , 2022-09-13T14:34:53.380 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/teuthology/orchestra/remote.py", line 96, in sh , proc = self.run(**kwargs) , File "/home/teuthworker/src/git.ceph.com_git_teuthology_b58f2c18636eb10faa77ed3614abd00cb85dfc2c/teuthology/orchestra/remote.py", line 510, in run Found 5 distinct failure reasons Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi130 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f860b69d9ecbf6a250d17b83c1778c894700357d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2 jobs: ['7030586', '7030590'] suites intersection: ['task/test_cephadm}'] suites union: ['agent/off', 'agent/on', 'mon_election/classic', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/rhel_8.4_container_tools_3.0', 'rados/cephadm/workunits/{0-distro/rhel_8.4_container_tools_rhel8', 'task/test_cephadm}'] Failure: Command crashed: 'CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 400000 --objects 10000 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 600 --op read 100 --op write 50 --op delete 50 --op copy_from 50 --op write_excl 50 --pool base' 1 jobs: ['7030585'] suites: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async', 'objectstore/bluestore-comp-zlib', 'rados', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'supported-random-distro$/{centos_8}}'] Failure: expected valgrind issues and found none 2 jobs: ['7030588', '7030583'] suites intersection: ['centos_latest}', 'rados/valgrind-leaks/{1-start'] suites union: ['2-inject-leak/mon', '2-inject-leak/osd', 'centos_latest}', 'rados/valgrind-leaks/{1-start'] Failure: Command failed on smithi003 with status 22: 'kubectl -n rook-ceph exec rook-ceph-tools-7564bb9799-9xb9l -- ceph orch apply osd --all-available-devices' 1 jobs: ['7030584'] suites: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '3-final', 'cluster/3-node', 'k8s/1.21', 'net/flannel', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/master}'] Failure: Command failed on smithi090 with status 1: 'kubectl apply -f https://docs.projectcalico.org/manifests/tigera-operator.yaml' 3 jobs: ['7030592', '7030587', '7030582'] suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', '3-final', 'k8s/1.21', 'rados/rook/smoke/{0-distro/ubuntu_20.04'] suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', '3-final', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/host', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}', 'rook/master}']