2021-02-04T00:05:40.953 INFO:root:teuthology version: 1.0.0-c383a14 2021-02-04T00:05:40.956 INFO:teuthology.results:subset: '176/300' 2021-02-04T00:05:40.956 INFO:teuthology.results:seed: '7615' 2021-02-04T05:01:57.614 INFO:root:teuthology version: 1.0.0-c383a14 2021-02-04T05:01:57.653 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-02-04T11:22:08.104 INFO:teuthology.results:Tests finished! gathering results... 2021-02-04T11:22:59.183 INFO:scrape:Found 189 jobs 2021-02-04T11:23:14.076 INFO:scrape:Found 6 distinct failure reasons 2021-02-04T11:23:14.080 INFO:scrape:Failure: Command failed on smithi116 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd require-osd-release pacific'" 2021-02-04T11:23:14.080 INFO:scrape:11 jobs: ['5854085', '5854118', '5854184', '5854167', '5854192', '5854218', '5854051', '5854153', '5854233', '5854078', '5854137'] 2021-02-04T11:23:14.081 INFO:scrape:suites intersection: ['clusters/{fixed-3', 'msgr-failures/few', 'openstack}'] 2021-02-04T11:23:14.081 INFO:scrape:suites union: ['clusters/{fixed-3', 'features/defaults', 'features/readbalance', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-snappy', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-hybrid', 'objectstore/bluestore-low-osd-mem-target', 'objectstore/bluestore-stupid', 'objectstore/filestore-xfs', 'openstack}', 'pool/ec-cache-pool', 'pool/ec-data-pool', 'pool/none', 'pool/replicated-data-pool', 'pool/small-cache-pool', 'rbd/encryption/{cache/none', 'rbd/encryption/{cache/writeback', 'rbd/encryption/{cache/writethrough', 'rbd/qemu/{cache/writearound', 'rbd/qemu/{cache/writeback', 'rbd/qemu/{cache/writethrough', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/qemu_fsstress}', 'workloads/qemu_xfstests_luks1}', 'workloads/qemu_xfstests_luks2}'] 2021-02-04T11:23:14.082 INFO:scrape: 2021-02-04T11:23:14.082 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/qemu_dynamic_features.sh 2021-02-04T11:23:14.082 INFO:scrape:1 jobs: ['5854145'] 2021-02-04T11:23:14.083 INFO:scrape:suites: ['clusters/{fixed-3', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'qemu/xfstests', 'rbd/maintenance/{base/install', 'supported-random-distro$/{ubuntu_latest}', 'workloads/dynamic_features_no_cache}'] 2021-02-04T11:23:14.083 INFO:scrape: 2021-02-04T11:23:14.084 INFO:scrape:Failure: Command failed (workunit test rbd/rbd_mirror_stress.sh) on smithi155 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=657132fb64e6d05fa44dc1ff3619f595d6de1c56 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS=\'\' RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh' 2021-02-04T11:23:14.085 INFO:scrape:5 jobs: ['5854136', '5854077', '5854169', '5854207', '5854154'] 2021-02-04T11:23:14.085 INFO:scrape:suites intersection: ['cluster/{2-node', 'msgr-failures/few', 'openstack}'] 2021-02-04T11:23:14.086 INFO:scrape:suites union: ['clients/mirror', 'clients/{mirror-extra', 'cluster/{2-node', 'mirror}', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-comp-zlib', 'objectstore/bluestore-hybrid', 'objectstore/filestore-xfs', 'openstack}', 'policy/none', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'rbd/mirror/{base/install', 'supported-random-distro$/{centos_8}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'workloads/rbd-mirror-stress-workunit-min-compat-client-octopus}', 'workloads/rbd-mirror-stress-workunit}'] 2021-02-04T11:23:14.086 INFO:scrape: 2021-02-04T11:23:14.087 INFO:scrape:Timeout 3h running clone.client.0/qa/workunits/rbd/test_librbd.sh 2021-02-04T11:23:14.088 INFO:scrape:1 jobs: ['5854071'] 2021-02-04T11:23:14.088 INFO:scrape:suites: ['clusters/{fixed-3', 'config/none', 'min-compat-client/octopus', 'msgr-failures/few', 'objectstore/bluestore-comp-lz4', 'openstack}', 'pool/replicated-data-pool', 'rbd/librbd/{cache/none', 'supported-random-distro$/{ubuntu_latest}', 'workloads/c_api_tests_with_journaling}'] 2021-02-04T11:23:14.089 INFO:scrape: 2021-02-04T11:23:14.089 INFO:scrape:Failure: unexpected end of data 2021-02-04T11:23:14.089 INFO:scrape:1 jobs: ['5854156'] 2021-02-04T11:23:14.090 INFO:scrape:suites: ['clusters/{fixed-3', 'features/defaults', 'msgr-failures/few', 'objectstore/bluestore-bitmap', 'openstack}', 'pool/small-cache-pool', 'rbd/qemu/{cache/writethrough', 'supported-random-distro$/{centos_8}', 'workloads/qemu_xfstests}'] 2021-02-04T11:23:14.091 INFO:scrape: 2021-02-04T11:23:14.091 INFO:scrape:Crash: Found coredumps on ubuntu@smithi162.front.sepia.ceph.com 2021-02-04T11:23:14.091 INFO:scrape:ceph version 17.0.0-421-g657132fb (657132fb64e6d05fa44dc1ff3619f595d6de1c56) quincy (dev) 1: /lib64/libpthread.so.0(+0x12dc0) [0x7f0d00488dc0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x7f0d01289c4b] 5: /usr/lib64/ceph/libceph-common.so.2(+0x275e14) [0x7f0d01289e14] 6: /lib64/librados.so.2(+0xec349) [0x7f0d0a8d1349] 7: /lib64/librados.so.2(+0xec575) [0x7f0d0a8d1575] 8: /lib64/librados.so.2(+0xed2b0) [0x7f0d0a8d22b0] 9: /lib64/librados.so.2(+0xf1d6a) [0x7f0d0a8d6d6a] 10: /lib64/librados.so.2(+0xf58c3) [0x7f0d0a8da8c3] 11: /lib64/librados.so.2(+0xbb64a) [0x7f0d0a8a064a] 12: (DispatchQueue::entry()+0x126a) [0x7f0d014c289a] 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0d01570161] 14: (Thread::_entry_func(void*)+0xd) [0x7f0d01370f8d] 15: /lib64/libpthread.so.0(+0x82de) [0x7f0d0047e2de] 16: clone() 2021-02-04T11:23:14.092 INFO:scrape:1 jobs: ['5854173'] 2021-02-04T11:23:14.092 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-comp-lz4', 'openstack}', 'policy/none', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{centos_8}', 'workloads/rbd-mirror-fsx-workunit}'] 2021-02-04T11:23:14.093 INFO:scrape: