Found 25 jobs Found 7 distinct failure reasons Dead: b'2020-10-09T21:34:53.308 DEBUG:teuthology.exit:Finished running handlers' 6 jobs: ['5510471', '5510464', '5510478', '5510468', '5510486', '5510482'] suites intersection: ['ceph', 'clusters/{fixed-2', 'openstack}', 'rados', 'thrashosds-health'] suites union: ['1-pg-log-overrides/normal_pg_log', '1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{default}', '2-recovery-overrides/{more-active-recovery}', '2-recovery-overrides/{more-async-recovery}', 'backoff/normal', 'backoff/peering', 'backoff/peering_and_degraded', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'crc-failures/default', 'd-balancer/crush-compat', 'd-balancer/on', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr-failures/few', 'msgr-failures/osd-delay', 'msgr/async', 'msgr/async-v1only', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'thrashers/careful', 'thrashers/default', 'thrashers/mapgap', 'thrashers/morepggrow', 'thrashers/pggrow', 'thrashosds-health', 'workloads/cache-snaps}', 'workloads/cache}', 'workloads/pool-snaps-few-objects}', 'workloads/radosbench-high-concurrency}', 'workloads/radosbench}', 'workloads/redirect}'] Failure: "2020-10-09T16:57:51.760959+0000 mon.a (mon.0) 494 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log 1 jobs: ['5510467'] suites: ['ceph', 'clusters/{fixed-2', 'd-thrash/none', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados/verify/{centos_latest', 'tasks/rados_cls_all', 'validater/valgrind}'] Crash: Command failed on smithi162 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'truncate --size 0 $TESTDIR/mnt.0/foo'" ceph version 16.0.0-5896-gff268523 (ff268523c2dccc40dc5cd034bded4b745bdb6816) pacific (dev) 1: () 2: (jaeger_tracing::child_span(char const*, std::unique_ptr > const&)+0x36) [0x56380f1a4cd6] 3: (PrimaryLogPG::finish_ctx(PrimaryLogPG::OpContext*, int, int)+0x1fd) [0x56380f30aabd] 4: (PrimaryLogPG::finish_promote(int, PrimaryLogPG::CopyResults*, std::shared_ptr)+0xf48) [0x56380f32f288] 5: (PromoteCallback::finish(boost::tuples::tuple)+0x57) [0x56380f3bdb07] 6: (PrimaryLogPG::process_copy_chunk(hobject_t, unsigned long, int)+0xbac) [0x56380f3421cc] 7: (C_Copyfrom::finish(int)+0xa6) [0x56380f3c0376] 8: (Context::complete(int)+0xd) [0x56380f22611d] 9: (Finisher::finisher_thread_entry()+0x1a5) [0x56380f816305] 10: () 11: clone() 1 jobs: ['5510479'] suites: ['mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/cache-fs-trunc', 'supported-random-distro$/{centos_8}}'] Failure: Test failure: test_pool_configuration (tasks.mgr.dashboard.test_pool.PoolTest) 1 jobs: ['5510476'] suites: ['debug/mgr', 'mon_election/classic', 'objectstore/bluestore-comp-lz4', 'rados/dashboard/{clusters/{2-node-mgr}', 'supported-random-distro$/{rhel_8}', 'tasks/dashboard}'] Failure: psutil.NoSuchProcess process no longer exists (pid=14787) 1 jobs: ['5510466'] suites: ['cluster/{12-osds', 'mon_election/connectivity', 'msgr-failures/osd-delay', 'objectstore/bluestore-comp-snappy', 'openstack}', 'rados', 'rados/thrash-erasure-code-big/{ceph', 'recovery-overrides/{more-partial-recovery}', 'supported-random-distro$/{rhel_8}', 'thrashers/morepggrow', 'thrashosds-health', 'workloads/ec-rados-plugin=jerasure-k=4-m=2}'] Timeout 3h running clone.client.0/qa/workunits/rados/test.sh 1 jobs: ['5510474'] suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-active-recovery}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/on', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr/async-v2only', 'objectstore/filestore-xfs', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{centos_8}', 'thrashers/none', 'thrashosds-health', 'workloads/rados_api_tests}'] Failure: Command failed (workunit test erasure-code/test-erasure-code.sh) on smithi105 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ff268523c2dccc40dc5cd034bded4b745bdb6816 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh' 1 jobs: ['5510484'] suites: ['rados/standalone/{mon_election/classic', 'supported-random-distro$/{centos_8}', 'workloads/erasure-code}']