2021-02-21T18:22:22.442 INFO:root:teuthology version: 1.1.0-9a6b849c 2021-02-21T18:22:22.444 INFO:teuthology.results:subset: '1/20' 2021-02-21T18:22:22.444 INFO:teuthology.results:seed: '4151' 2021-02-21T18:49:49.953 INFO:root:teuthology version: 1.1.0-9a6b849c 2021-02-21T18:49:49.984 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-02-21T20:20:51.720 INFO:teuthology.results:Tests finished! gathering results... 2021-02-21T20:21:11.896 INFO:scrape:Found 29 jobs 2021-02-21T20:21:12.156 INFO:scrape:Found 1 distinct failure reasons 2021-02-21T20:21:12.156 INFO:scrape:Crash: "2021-02-21T18:48:16.507014+0000 mon.a (mon.0) 421 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log 2021-02-21T20:21:12.157 INFO:scrape:ceph version 17.0.0-971-gf57f6ff0 (f57f6ff08fac47da951bb8b6ee6407b3af5e5e37) quincy (dev) 1: /lib64/libpthread.so.0(+0x12dc0) [0x7f049557bdc0] 2: gsignal() 3: abort() 4: /lib64/libstdc++.so.6(+0x9006b) [0x7f0494b9e06b] 5: /lib64/libstdc++.so.6(+0x9650c) [0x7f0494ba450c] 6: /lib64/libstdc++.so.6(+0x95529) [0x7f0494ba3529] 7: __gxx_personality_v0() 8: /lib64/libgcc_s.so.1(+0x10b13) [0x7f0494584b13] 9: _Unwind_Resume() 10: ceph-osd(+0x566fe2) [0x55a4e80fafe2] 11: (ShardedThreadPool::WorkThreadSharded::entry()+0x14) [0x55a4e888a224] 12: (Thread::_entry_func(void*)+0xd) [0x55a4e887908d] 13: /lib64/libpthread.so.0(+0x82de) [0x7f04955712de] 14: clone() 2021-02-21T20:21:12.157 INFO:scrape:1 jobs: ['5900609'] 2021-02-21T20:21:12.158 INFO:scrape:suites: ['clients/mirror', 'cluster/{2-node', 'msgr-failures/few', 'objectstore/bluestore-comp-snappy', 'openstack}', 'policy/simple', 'rbd-mirror/four-per-cluster', 'rbd/mirror-thrash/{base/install', 'supported-random-distro$/{centos_8}', 'workloads/rbd-mirror-snapshot-stress-workunit-minimum}'] 2021-02-21T20:21:12.158 INFO:scrape: 2021-02-21T20:21:12.159 INFO:teuthology.results:Sending results to to.my.trociny@gmail.com: Test Run: trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 28 Fail ================================================================= [5900609] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum} ----------------------------------------------------------------- time: 01:32:45 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900609/ log: http://qa-proxy.ceph.com/teuthology/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900609/ "2021-02-21T18:48:16.507014+0000 mon.a (mon.0) 421 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN)" in cluster log Pass ================================================================= [5900608] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-journaling} time: 00:17:10 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900608/ [5900610] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-minimum} time: 00:18:45 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900610/ [5900611] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit} time: 01:22:03 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900611/ [5900612] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{centos_8} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus} time: 00:58:25 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900612/ [5900613] rbd/singleton/{all/rbd_mirror objectstore/bluestore-bitmap openstack supported-random-distro$/{ubuntu_latest}} time: 00:41:20 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900613/ [5900614] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-stress-workunit} time: 01:17:49 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900614/ [5900615] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-config-key} time: 00:21:08 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900615/ [5900616] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-min-compat-client-octopus} time: 00:22:54 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900616/ [5900617] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-journal-workunit} time: 00:29:31 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900617/ [5900618] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-policy-none} time: 00:22:46 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900618/ [5900619] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} time: 01:16:12 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900619/ [5900620] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-low-osd-mem-target validator/memcheck workloads/rbd_mirror} time: 01:01:49 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900620/ [5900621] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} workloads/rbd-mirror-workunit-policy-simple} time: 00:23:05 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900621/ [5900622] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-bootstrap-workunit} time: 00:08:32 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900622/ [5900623] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-fast-diff} time: 01:36:11 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900623/ [5900624] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_8} workloads/rbd-mirror-ha-workunit} time: 00:16:42 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900624/ [5900625] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum} time: 00:57:25 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900625/ [5900626] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-low-osd-mem-target supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-exclusive-lock} time: 00:18:01 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900626/ [5900627] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/none rbd-mirror/four-per-cluster supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-stress-workunit-minimum} time: 01:33:24 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900627/ [5900628] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-fast-diff} time: 00:17:29 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900628/ [5900629] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/filestore-xfs supported-random-distro$/{rhel_8} workloads/rbd-mirror-snapshot-workunit-journaling} time: 00:19:18 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900629/ [5900630] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-fsx-workunit} time: 01:18:47 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900630/ [5900631] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_8} workloads/rbd-mirror-snapshot-workunit-minimum} time: 00:18:29 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900631/ [5900632] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-stress-workunit} time: 01:21:31 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900632/ [5900633] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_8} workloads/rbd-mirror-stress-workunit-min-compat-client-octopus} time: 01:22:45 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900633/ [5900634] rbd/singleton/{all/rbd_mirror objectstore/filestore-xfs openstack supported-random-distro$/{centos_8}} time: 00:43:56 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900634/ [5900635] rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_8} workloads/rbd-mirror-journal-workunit} time: 00:33:59 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900635/ [5900636] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-config-key} time: 00:20:36 info: http://pulpito.ceph.com/trociny-2021-02-21_18:20:00-rbd-wip-mgolub-testing-distro-basic-smithi/5900636/