2020-09-02T10:49:18.898 INFO:root:teuthology version: 1.0.0-f1fe128 2020-09-02T10:49:18.899 INFO:teuthology.results:seed: '3629' 2020-09-02T12:36:07.972 INFO:root:teuthology version: 1.0.0-f1fe128 2020-09-02T12:36:08.006 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-09-02T22:39:25.711 INFO:teuthology.results:Tests finished! gathering results... 2020-09-02T22:39:36.150 INFO:scrape:Found 96 jobs 2020-09-02T22:39:40.023 INFO:scrape:Found 4 distinct failure reasons 2020-09-02T22:39:40.025 INFO:scrape:Failure: Found coredumps on ubuntu@smithi037.front.sepia.ceph.com 2020-09-02T22:39:40.026 INFO:scrape:2 jobs: ['5391989', '5391940'] 2020-09-02T22:39:40.026 INFO:scrape:suites intersection: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'tasks/rbd_concurrent}'] 2020-09-02T22:39:40.027 INFO:scrape:suites union: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'msgr-failures/few', 'msgr-failures/many', 'tasks/rbd_concurrent}'] 2020-09-02T22:39:40.028 INFO:scrape: 2020-09-02T22:39:40.028 INFO:scrape:Failure: "2020-09-02T12:48:30.754906+0000 mon.a (mon.0) 31 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log 2020-09-02T22:39:40.028 INFO:scrape:5 jobs: ['5392001', '5391984', '5391948', '5391983', '5392005'] 2020-09-02T22:39:40.029 INFO:scrape:suites intersection: ['conf'] 2020-09-02T22:39:40.030 INFO:scrape:suites union: ['ceph/ceph', 'clusters/3-node', 'clusters/fixed-3', 'conf', 'features/no-object-map', 'install/ceph', 'krbd/fsx/{ceph/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'krbd/rbd/{bluestore-bitmap', 'krbd/wac/wac/{bluestore-bitmap', 'msgr-failures/many', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'randomized-striping-off}', 'striping/default/{msgr-failures/many', 'tasks/fsx-3-client}', 'tasks/rbd_map_snapshot_io}', 'tasks/rbd_workunit_suites_fsstress}', 'tasks/wac', 'verify/many-resets}'] 2020-09-02T22:39:40.031 INFO:scrape: 2020-09-02T22:39:40.031 INFO:scrape:Crash: Command failed (workunit test rbd/huge-tickets.sh) on smithi193 with status 110: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=568c46044525e82fe6e50ab9cc37c79f2246836a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/huge-tickets.sh' 2020-09-02T22:39:40.031 INFO:scrape:ceph version 16.0.0-4878-g90e5434905 (90e54349056202e5442a8e57c5752da4ea85225b) pacific (dev) 1: (()+0x128a0) [0x7fdd2bc278a0] 2: (gsignal()+0xc7) [0x7fdd2ad1ff47] 3: (abort()+0x141) [0x7fdd2ad218b1] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x19c) [0x7fdd2cdade1a] 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x7fdd2cdadfa4] 6: (ProtocolV2::write(std::__cxx11::basic_string, std::allocator > const&, CtFun&, ceph::buffer::v15_2_0::list&)+0x1ac) [0x7fdd2d07637c] 7: (Ct* ProtocolV2::write(std::__cxx11::basic_string, std::allocator > const&, CtFun&, ceph::msgr::v2::AuthDoneFrame&)+0x18b) [0x7fdd2d0927db] 8: (ProtocolV2::_handle_auth_request(ceph::buffer::v15_2_0::list&, bool)+0x34a) [0x7fdd2d082a5a] 9: (ProtocolV2::handle_auth_request_more(ceph::buffer::v15_2_0::list&)+0x23e) [0x7fdd2d0837fe] 10: (ProtocolV2::handle_frame_payload()+0xfb) [0x7fdd2d08bc8b] 11: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x7fdd2d08bff0] 12: (ProtocolV2::_handle_read_frame_epilogue_main()+0x69) [0x7fdd2d08c159] 13: (ProtocolV2::_handle_read_frame_segment()+0x121) [0x7fdd2d08c511] 14: (ProtocolV2::handle_read_frame_segment(std::unique_ptr&&, int)+0xd7) [0x7fdd2d08d6f7] 15: (ProtocolV2::run_continuation(Ct&)+0x34) [0x7fdd2d075c64] 16: (AsyncConnection::process()+0x5fc) [0x7fdd2d040dfc] 17: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0x7dd) [0x7fdd2d096b5d] 18: (()+0x554920) [0x7fdd2d09e920] 19: (std::function::operator()() const+0x32) [0x561fa52b6e1c] 20: (void std::__invoke_impl>(std::__invoke_other, std::function&&)+0x20) [0x561fa5650511] 21: (std::__invoke_result>::type std::__invoke>(std::function&&)+0x35) [0x561fa564f68d] 22: (decltype (__invoke((_S_declval<0ul>)())) std::thread::_Invoker > >::_M_invoke<0ul>(std::_Index_tuple<0ul>)+0x28) [0x561fa565154c] 23: (std::thread::_Invoker > >::operator()()+0x2c) [0x561fa5651508] 24: (std::thread::_State_impl > > >::_M_run()+0x1c) [0x561fa56514d8] 25: (()+0xbd6df) [0x7fdd2b7456df] 26: (()+0x76db) [0x7fdd2bc1c6db] 27: (clone()+0x3f) [0x7fdd2ae02a3f] 2020-09-02T22:39:40.032 INFO:scrape:2 jobs: ['5391943', '5391992'] 2020-09-02T22:39:40.032 INFO:scrape:suites intersection: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'tasks/rbd_huge_tickets}'] 2020-09-02T22:39:40.033 INFO:scrape:suites union: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'msgr-failures/few', 'msgr-failures/many', 'tasks/rbd_huge_tickets}'] 2020-09-02T22:39:40.033 INFO:scrape: 2020-09-02T22:39:40.034 INFO:scrape:Failure: Command failed on smithi162 with status 4: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludec0t52wa3 -r -- '-g auto -g blockdev -x clone'" 2020-09-02T22:39:40.036 INFO:scrape:2 jobs: ['5391964', '5391915'] 2020-09-02T22:39:40.036 INFO:scrape:suites intersection: ['conf', 'krbd/singleton/{bluestore-bitmap', 'tasks/rbd_xfstests}'] 2020-09-02T22:39:40.037 INFO:scrape:suites union: ['conf', 'krbd/singleton/{bluestore-bitmap', 'msgr-failures/few', 'msgr-failures/many', 'tasks/rbd_xfstests}'] 2020-09-02T22:39:40.037 INFO:scrape: 2020-09-02T22:39:40.043 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/ failed: 11 dead: 0 running: 0 waiting: 0 queued: 0 passed: 85 Fail ================================================================= [5391915] krbd/singleton/{bluestore-bitmap conf msgr-failures/few tasks/rbd_xfstests} ----------------------------------------------------------------- time: 10:24:57 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391915/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391915/ sentry: https://sentry.ceph.com/sepia/teuthology/?query=6d2ccba0d6b1489e8dad13c625039374 Command failed on smithi125 with status 3: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude_krduhm8 -r -- '-g auto -g blockdev -x clone'" [5391940] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_concurrent} ----------------------------------------------------------------- time: 00:14:56 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391940/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391940/ Found coredumps on ubuntu@smithi183.front.sepia.ceph.com [5391943] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_huge_tickets} ----------------------------------------------------------------- time: 00:14:56 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391943/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391943/ sentry: https://sentry.ceph.com/sepia/teuthology/?query=55531723a52842b1829203ff12470dd6 Command failed (workunit test rbd/huge-tickets.sh) on smithi193 with status 110: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=568c46044525e82fe6e50ab9cc37c79f2246836a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/huge-tickets.sh' [5391948] krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} ----------------------------------------------------------------- time: 00:25:40 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391948/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391948/ "2020-09-02T12:17:50.760873+0000 mon.b (mon.0) 32 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [5391964] krbd/singleton/{bluestore-bitmap conf msgr-failures/many tasks/rbd_xfstests} ----------------------------------------------------------------- time: 09:52:12 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391964/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391964/ sentry: https://sentry.ceph.com/sepia/teuthology/?query=70c27db46c834c76921e3ea33baaef91 Command failed on smithi162 with status 4: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludec0t52wa3 -r -- '-g auto -g blockdev -x clone'" [5391983] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:28:08 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391983/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391983/ "2020-09-02T12:53:16.810240+0000 mon.b (mon.0) 31 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [5391984] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_fsstress} ----------------------------------------------------------------- time: 00:13:59 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391984/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391984/ "2020-09-02T12:54:42.291156+0000 mon.b (mon.0) 41 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [5391989] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_concurrent} ----------------------------------------------------------------- time: 00:15:37 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391989/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391989/ Found coredumps on ubuntu@smithi037.front.sepia.ceph.com [5391992] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_huge_tickets} ----------------------------------------------------------------- time: 00:14:51 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391992/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391992/ sentry: https://sentry.ceph.com/sepia/teuthology/?query=f59534c8c5f94996bf0b671f7563bf50 Command failed (workunit test rbd/huge-tickets.sh) on smithi105 with status 110: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=568c46044525e82fe6e50ab9cc37c79f2246836a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/huge-tickets.sh' [5392001] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_map_snapshot_io} ----------------------------------------------------------------- time: 00:09:44 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5392001/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5392001/ "2020-09-02T12:48:30.754906+0000 mon.a (mon.0) 31 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log [5392005] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:31:00 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5392005/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5392005/ "2020-09-02T12:58:59.129264+0000 mon.a (mon.0) 32 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY)" in cluster log Pass ================================================================= [5391911] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_deep_flatten} time: 00:08:57 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391911/ [5391912] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:25:59 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391912/ [5391913] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_fio} time: 00:41:49 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391913/ [5391914] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_data_pool} time: 00:20:49 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391914/ [5391916] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/backoff thrashosds-health workloads/rbd_fio} time: 01:13:41 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391916/ [5391917] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} time: 00:06:42 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391917/ [5391918] krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_pages_required} time: 00:06:36 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391918/ [5391919] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_exclusive_option} time: 00:11:08 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391919/ [5391920] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:26:03 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391920/ [5391921] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_kernel_untar_build} time: 00:15:27 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391921/ [5391922] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_fallocate} time: 00:08:48 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391922/ [5391923] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:26:54 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391923/ [5391924] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_huge_osdmap} time: 00:09:54 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391924/ [5391925] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_dbench} time: 00:46:32 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391925/ [5391926] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_discard} time: 00:08:44 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391926/ [5391927] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:25:46 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391927/ [5391928] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:16:04 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391928/ [5391929] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_latest_osdmap_on_map} time: 00:12:56 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391929/ [5391930] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:33:29 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391930/ [5391931] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_ffsb} time: 01:23:02 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391931/ [5391932] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_namespaces} time: 00:11:50 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391932/ [5391933] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_udev_enumerate} time: 00:26:00 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391933/ [5391934] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:27:10 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391934/ [5391935] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_fsstress} time: 00:08:52 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391935/ [5391936] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_udev_netlink_enobufs} time: 00:14:15 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391936/ [5391937] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_huge_image} time: 00:06:35 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391937/ [5391938] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:26:53 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391938/ [5391939] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/pggrow thrashosds-health workloads/rbd_fio} time: 01:20:28 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391939/ [5391941] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} time: 00:08:36 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391941/ [5391942] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:27:44 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391942/ [5391944] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:24:08 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391944/ [5391945] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_fsx} time: 00:16:41 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391945/ [5391946] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_image_read} time: 00:08:37 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391946/ [5391947] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} time: 00:07:25 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391947/ [5391949] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_kernel} time: 00:08:03 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391949/ [5391950] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_modprobe} time: 00:06:52 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391950/ [5391951] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:24:57 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391951/ [5391952] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:53:48 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391952/ [5391953] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_iozone} time: 00:11:46 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391953/ [5391954] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_map_snapshot_io} time: 00:07:54 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391954/ [5391955] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:27:29 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391955/ [5391956] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_map_unmap} time: 00:12:46 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391956/ [5391957] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_trivial_sync} time: 00:08:22 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391957/ [5391958] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:28:36 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391958/ [5391959] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_simple_big} time: 00:33:56 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391959/ [5391960] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_msgr_segments} time: 00:06:50 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391960/ [5391961] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:28:39 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391961/ [5391962] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_fio} time: 00:39:20 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391962/ [5391963] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_data_pool} time: 00:26:54 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391963/ [5391965] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 02:10:52 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391965/ [5391966] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_exclusive_option} time: 00:10:21 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391966/ [5391967] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:28:25 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391967/ [5391968] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_kernel_untar_build} time: 00:15:04 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391968/ [5391969] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_fallocate} time: 00:09:50 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391969/ [5391970] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:32:05 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391970/ [5391971] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_huge_osdmap} time: 00:09:41 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391971/ [5391972] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_dbench} time: 00:48:56 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391972/ [5391973] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_parent_overlap} time: 00:07:14 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391973/ [5391974] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:24:27 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391974/ [5391975] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} time: 01:28:31 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391975/ [5391976] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_latest_osdmap_on_map} time: 00:13:17 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391976/ [5391977] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:31:30 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391977/ [5391978] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_ffsb} time: 01:13:40 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391978/ [5391979] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_namespaces} time: 00:11:09 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391979/ [5391980] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} time: 00:07:06 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391980/ [5391981] krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} time: 00:21:34 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391981/ [5391982] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_udev_enumerate} time: 02:07:45 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391982/ [5391985] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_udev_netlink_enobufs} time: 00:13:54 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391985/ [5391986] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_read_only} time: 00:07:58 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391986/ [5391987] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:29:01 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391987/ [5391988] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:32:23 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391988/ [5391990] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} time: 00:08:54 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391990/ [5391991] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:32:26 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391991/ [5391993] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:32:56 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391993/ [5391994] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_fsx} time: 00:17:04 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391994/ [5391995] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_image_read} time: 00:08:57 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391995/ [5391996] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_kernel} time: 00:08:20 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391996/ [5391997] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_whole_object_zeroout} time: 00:07:29 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391997/ [5391998] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:23:14 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391998/ [5391999] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/upmap thrashosds-health workloads/rbd_fio} time: 01:25:05 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5391999/ [5392000] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_iozone} time: 00:11:15 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5392000/ [5392002] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:30:44 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5392002/ [5392003] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_map_unmap} time: 00:13:31 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5392003/ [5392004] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_trivial_sync} time: 00:08:21 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5392004/ [5392006] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_simple_big} time: 00:35:06 info: http://pulpito.ceph.com/teuthology-2020-08-31_03:20:03-krbd-master-testing-basic-smithi/5392006/