2020-12-11T04:54:26.277 INFO:root:teuthology version: 1.0.0-a4989a90 2020-12-11T04:54:26.279 INFO:teuthology.results:subset: '7/10' 2020-12-11T04:54:26.279 INFO:teuthology.results:seed: '9057' 2020-12-11T05:35:14.288 INFO:root:teuthology version: 1.0.0-a4989a90 2020-12-11T05:35:14.333 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-12-11T10:37:20.333 INFO:teuthology.results:Tests finished! gathering results... 2020-12-11T10:37:29.537 INFO:scrape:Found 50 jobs 2020-12-11T10:37:29.687 INFO:scrape:Found 1 distinct failure reasons 2020-12-11T10:37:29.688 INFO:scrape:Failure: "2020-12-11T05:34:19.079444+0000 osd.1 (osd.1) 3 : cluster [WRN] slow request osd_op(client.4265.0:777 3.2 3:60da0696:::rbd_data.107c220f24cd.00000000000000e6:5 [read 2453504~4096 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e29) initiated 2020-12-11T05:33:48.997155+0000 currently reached pg" in cluster log 2020-12-11T10:37:29.689 INFO:scrape:1 jobs: ['5699836'] 2020-12-11T10:37:29.689 INFO:scrape:suites: ['clusters/{fixed-1', 'features/journaling', 'msgr-failures/few', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'pool/small-cache-pool', 'rbd/cli/{base/install', 'supported-random-distro$/{rhel_8}', 'workloads/rbd_cli_migration}'] 2020-12-11T10:37:29.689 INFO:scrape: 2020-12-11T10:37:29.692 INFO:teuthology.results:Sending results to to.my.trociny@gmail.com: Test Run: trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 49 Fail ================================================================= [5699836] rbd/cli/{base/install clusters/{fixed-1 openstack} features/journaling msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/small-cache-pool supported-random-distro$/{rhel_8} workloads/rbd_cli_migration} ----------------------------------------------------------------- time: 00:35:27 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699836/ log: http://qa-proxy.ceph.com/teuthology/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699836/ "2020-12-11T05:34:19.079444+0000 osd.1 (osd.1) 3 : cluster [WRN] slow request osd_op(client.4265.0:777 3.2 3:60da0696:::rbd_data.107c220f24cd.00000000000000e6:5 [read 2453504~4096 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected e29) initiated 2020-12-11T05:33:48.997155+0000 currently reached pg" in cluster log Pass ================================================================= [5699812] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-bitmap pool/replicated-data-pool supported-random-distro$/{centos_8} workloads/c_api_tests_with_defaults} time: 00:25:14 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699812/ [5699813] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-lz4 pool/small-cache-pool supported-random-distro$/{ubuntu_latest} workloads/c_api_tests_with_journaling} time: 00:39:22 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699813/ [5699814] rbd/qemu/{cache/writearound clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/filestore-xfs pool/replicated-data-pool supported-random-distro$/{ubuntu_latest} workloads/qemu_xfstests} time: 01:23:24 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699814/ [5699815] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/none min-compat-client/default msgr-failures/few objectstore/bluestore-comp-snappy pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/fsx} time: 00:33:17 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699815/ [5699816] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-zlib pool/none supported-random-distro$/{ubuntu_latest} workloads/python_api_tests} time: 00:18:08 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699816/ [5699817] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/default msgr-failures/few objectstore/bluestore-comp-zstd pool/replicated-data-pool supported-random-distro$/{ubuntu_latest} workloads/python_api_tests_with_defaults} time: 00:18:32 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699817/ [5699818] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/none min-compat-client/octopus msgr-failures/few objectstore/bluestore-hybrid pool/small-cache-pool supported-random-distro$/{ubuntu_latest} workloads/python_api_tests_with_journaling} time: 00:19:16 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699818/ [5699819] rbd/immutable-object-cache/{clusters/{fix-2 openstack} pool/ceph_and_immutable_object_cache supported-random-distro$/{rhel_8} workloads/c_api_tests_with_defaults} time: 00:24:53 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699819/ [5699820] rbd/qemu/{cache/writeback clusters/{fixed-3 openstack} features/journaling msgr-failures/few objectstore/bluestore-bitmap pool/small-cache-pool supported-random-distro$/{ubuntu_latest} workloads/qemu_bonnie} time: 01:11:47 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699820/ [5699821] rbd/singleton-bluestore/{all/issue-20295 objectstore/bluestore-bitmap openstack supported-random-distro$/{ubuntu_latest}} time: 00:19:12 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699821/ [5699822] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/rbd_fio} time: 01:21:49 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699822/ [5699823] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/octopus msgr-failures/few objectstore/bluestore-stupid pool/none supported-random-distro$/{ubuntu_latest} workloads/c_api_tests} time: 00:23:44 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699823/ [5699824] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/none min-compat-client/default msgr-failures/few objectstore/filestore-xfs pool/replicated-data-pool supported-random-distro$/{centos_8} workloads/c_api_tests_with_defaults} time: 00:24:27 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699824/ [5699825] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/octopus msgr-failures/few objectstore/bluestore-bitmap pool/small-cache-pool supported-random-distro$/{ubuntu_latest} workloads/c_api_tests_with_journaling} time: 00:48:42 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699825/ [5699826] rbd/qemu/{cache/writethrough clusters/{fixed-3 openstack} features/readbalance msgr-failures/few objectstore/bluestore-comp-lz4 pool/ec-cache-pool supported-random-distro$/{rhel_8} workloads/qemu_fsstress} time: 00:13:30 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699826/ [5699827] rbd/singleton/{all/rbd_tasks objectstore/filestore-xfs openstack supported-random-distro$/{ubuntu_latest}} time: 00:07:22 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699827/ [5699828] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/default msgr-failures/few objectstore/bluestore-comp-lz4 pool/ec-data-pool supported-random-distro$/{centos_8} workloads/fsx} time: 00:26:52 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699828/ [5699829] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/none min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-snappy pool/none supported-random-distro$/{rhel_8} workloads/python_api_tests} time: 00:20:10 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699829/ [5699830] rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} thrashers/cache thrashosds-health workloads/rbd_fsx_copy_on_read} time: 00:14:41 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699830/ [5699831] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-comp-zlib pool/replicated-data-pool supported-random-distro$/{rhel_8} workloads/python_api_tests_with_defaults} time: 00:20:17 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699831/ [5699832] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-zstd pool/small-cache-pool supported-random-distro$/{ubuntu_latest} workloads/python_api_tests_with_journaling} time: 00:19:13 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699832/ [5699833] rbd/qemu/{cache/none clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-comp-snappy pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/qemu_xfstests} time: 04:54:52 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699833/ [5699834] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/none min-compat-client/default msgr-failures/few objectstore/bluestore-hybrid pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/rbd_fio} time: 01:24:59 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699834/ [5699835] rbd/basic/{base/install cachepool/small clusters/{fixed-1 openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_8} tasks/rbd_cls_tests} time: 00:10:50 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699835/ [5699837] rbd/cli_v1/{base/install clusters/{fixed-1 openstack} features/format-1 msgr-failures/few objectstore/bluestore-comp-snappy pool/none supported-random-distro$/{ubuntu_latest} workloads/rbd_cli_import_export} time: 00:06:52 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699837/ [5699838] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/octopus msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/none supported-random-distro$/{ubuntu_latest} workloads/c_api_tests} time: 00:24:08 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699838/ [5699839] rbd/nbd/{base/install cluster/{fixed-3 openstack} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_latest} thrashers/cache thrashosds-health workloads/rbd_nbd} time: 00:09:19 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699839/ [5699840] rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} objectstore/bluestore-comp-snappy validator/memcheck workloads/python_api_tests_with_defaults} time: 00:32:40 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699840/ [5699841] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/default msgr-failures/few objectstore/bluestore-stupid pool/replicated-data-pool supported-random-distro$/{rhel_8} workloads/c_api_tests_with_defaults} time: 00:25:54 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699841/ [5699842] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/none min-compat-client/octopus msgr-failures/few objectstore/filestore-xfs pool/small-cache-pool supported-random-distro$/{centos_8} workloads/c_api_tests_with_journaling} time: 00:28:05 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699842/ [5699843] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-bitmap pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/fsx} time: 00:27:15 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699843/ [5699844] rbd/qemu/{cache/writearound clusters/{fixed-3 openstack} features/journaling msgr-failures/few objectstore/bluestore-comp-zlib pool/none supported-random-distro$/{centos_8} workloads/qemu_bonnie} time: 01:11:40 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699844/ [5699845] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-lz4 pool/none supported-random-distro$/{centos_8} workloads/python_api_tests} time: 00:19:57 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699845/ [5699846] rbd/mirror/{base/install clients/{mirror-extra mirror} cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-workunit-config-key} time: 00:20:44 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699846/ [5699847] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/none min-compat-client/default msgr-failures/few objectstore/bluestore-comp-snappy pool/replicated-data-pool supported-random-distro$/{rhel_8} workloads/python_api_tests_with_defaults} time: 00:20:18 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699847/ [5699848] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-zlib pool/small-cache-pool supported-random-distro$/{ubuntu_latest} workloads/python_api_tests_with_journaling} time: 00:19:39 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699848/ [5699849] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/default msgr-failures/few objectstore/bluestore-comp-zstd pool/ec-data-pool supported-random-distro$/{centos_8} workloads/rbd_fio} time: 01:20:48 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699849/ [5699850] rbd/qemu/{cache/writeback clusters/{fixed-3 openstack} features/readbalance msgr-failures/few objectstore/bluestore-comp-zstd pool/replicated-data-pool supported-random-distro$/{ubuntu_latest} workloads/qemu_fsstress} time: 00:11:49 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699850/ [5699851] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/none min-compat-client/octopus msgr-failures/few objectstore/bluestore-hybrid pool/none supported-random-distro$/{centos_8} workloads/c_api_tests} time: 00:24:32 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699851/ [5699852] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-low-osd-mem-target pool/replicated-data-pool supported-random-distro$/{rhel_8} workloads/c_api_tests_with_defaults} time: 00:26:15 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699852/ [5699853] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/octopus msgr-failures/few objectstore/bluestore-stupid pool/small-cache-pool supported-random-distro$/{ubuntu_latest} workloads/c_api_tests_with_journaling} time: 00:35:52 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699853/ [5699854] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/none min-compat-client/default msgr-failures/few objectstore/filestore-xfs pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/fsx} time: 00:23:28 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699854/ [5699855] rbd/qemu/{cache/writethrough clusters/{fixed-3 openstack} features/defaults msgr-failures/few objectstore/bluestore-hybrid pool/small-cache-pool supported-random-distro$/{centos_8} workloads/qemu_xfstests} time: 04:41:43 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699855/ [5699856] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/octopus msgr-failures/few objectstore/bluestore-bitmap pool/none supported-random-distro$/{centos_8} workloads/python_api_tests} time: 00:20:06 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699856/ [5699857] rbd/librbd/{cache/writearound clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/default msgr-failures/few objectstore/bluestore-comp-lz4 pool/replicated-data-pool supported-random-distro$/{ubuntu_latest} workloads/python_api_tests_with_defaults} time: 00:18:34 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699857/ [5699858] rbd/librbd/{cache/writeback clusters/{fixed-3 openstack} config/none min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-snappy pool/small-cache-pool supported-random-distro$/{rhel_8} workloads/python_api_tests_with_journaling} time: 00:21:24 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699858/ [5699859] rbd/librbd/{cache/writethrough clusters/{fixed-3 openstack} config/permit-partial-discard min-compat-client/default msgr-failures/few objectstore/bluestore-comp-zlib pool/ec-data-pool supported-random-distro$/{centos_8} workloads/rbd_fio} time: 01:21:52 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699859/ [5699860] rbd/cli/{base/install clusters/{fixed-1 openstack} features/layering msgr-failures/few objectstore/bluestore-stupid pool/ec-data-pool supported-random-distro$/{rhel_8} workloads/rbd_cli_generic} time: 00:24:02 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699860/ [5699861] rbd/librbd/{cache/none clusters/{fixed-3 openstack} config/copy-on-read min-compat-client/octopus msgr-failures/few objectstore/bluestore-comp-zstd pool/none supported-random-distro$/{ubuntu_latest} workloads/c_api_tests} time: 00:25:19 info: http://pulpito.ceph.com/trociny-2020-12-11_04:53:41-rbd-wip-mgolub-testing-distro-basic-smithi/5699861/