Found 358 jobs Misunderstood line: Binary file /home/teuthworker/archive/teuthology-2020-09-19_07:01:02-rados-master-distro-basic-smithi/5448520/teuthology.log matches Misunderstood line: Ignoring malparsed backtrace: ceph version 16.0.0-5752-g47e7cd0f (47e7cd0f65780c9fdd80ca269e4d5761a75c7938) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)index user version cas cmpomap queue 2pc_queue fifo+0x158) [0x7f8402e224f8] , 2: () Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-09-19T08:55:24.694 INFO:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json , 2020-09-19T08:55:25.063 INFO:teuthology.orchestra.run.smithi155.stdout: Ignoring malparsed backtrace: 2020-09-19T08:56:47.059 INFO:teuthology.orchestra.run:waiting for 300 , 2020-09-19T08:56:47.141 INFO:tasks.ceph.osd.5:Stopped , 2020-09-19T08:56:47.141 INFO:teuthology.misc:Shutting down mgr daemons... Ignoring malparsed backtrace: 2020-09-19T08:59:10.142 INFO:teuthology.orchestra.run.smithi119.stdout:^- hv03.front.sepia.ceph.com 3 6 377 6 -5046us[-5048us] +/- 99ms , 2020-09-19T08:59:10.142 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 5 +537us[ +537us] +/- 87ms , 2020-09-19T08:59:10.143 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait , self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 182, in _raise_for_status Ignoring malparsed backtrace: ceph version 16.0.0-5752-g47e7cd0f (47e7cd0f65780c9fdd80ca269e4d5761a75c7938) pacific (dev) , 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)index user version cas cmpomap queue 2pc_queue fifo+0x158) [0x7f8402e224f8] , 2: () Ignoring malparsed backtrace: instructed osd(s) 5 to deep-scrub , 2020-09-19T08:55:24.694 INFO:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early pg dump --format=json , 2020-09-19T08:55:25.063 INFO:teuthology.orchestra.run.smithi155.stdout: Ignoring malparsed backtrace: 2020-09-19T08:56:47.059 INFO:teuthology.orchestra.run:waiting for 300 , 2020-09-19T08:56:47.141 INFO:tasks.ceph.osd.5:Stopped , 2020-09-19T08:56:47.141 INFO:teuthology.misc:Shutting down mgr daemons... Ignoring malparsed backtrace: 2020-09-19T08:59:10.142 INFO:teuthology.orchestra.run.smithi119.stdout:^- hv03.front.sepia.ceph.com 3 6 377 6 -5046us[-5048us] +/- 99ms , 2020-09-19T08:59:10.142 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv04.front.sepia.ceph.com 3 6 377 5 +537us[ +537us] +/- 87ms , 2020-09-19T08:59:10.143 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab Ignoring malparsed backtrace: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait , self._raise_for_status() , File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 182, in _raise_for_status Found 3 distinct failure reasons Failure: "2020-09-19T08:14:16.438147+0000 mds.a (mds.0) 1 : cluster [WRN] evicting unresponsive client smithi144:x (4775), after 304.971 seconds" in cluster log 1 jobs: ['5448262'] suites: ['1-install/nautilus', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{rbd-cls', '5-workload/{radosbench', '6-finish-upgrade', '7-octopus', '8-workload/{rbd-python', 'bluestore-bitmap', 'mon_election/classic', 'rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack', 'rbd-import-export', 'rbd_api}', 'readwrite', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_latest}'] Failure: Command failed (workunit test scrub/osd-scrub-repair.sh) on smithi136 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=47e7cd0f65780c9fdd80ca269e4d5761a75c7938 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-repair.sh' 1 jobs: ['5448594'] suites: ['rados/standalone/{mon_election/connectivity', 'supported-random-distro$/{ubuntu_latest}', 'workloads/scrub}'] Valgrind: 3 jobs: ['5448520', '5448405', '5448582'] suites intersection: ['rados'] suites union: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/few', 'msgr-failures/many', 'msgr/async', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/singleton/{all/thrash_cache_writeback_proxy_none', 'rados/verify/{centos_latest', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}}', 'tasks/rados_api_tests', 'tasks/rados_python}', 'validater/valgrind}']