2020-12-25T20:13:03.642 INFO:root:teuthology version: 1.0.0-8e49dda4 2020-12-25T20:13:03.643 INFO:teuthology.results:seed: '2801' 2020-12-25T21:26:55.596 INFO:root:teuthology version: 1.0.0-8e49dda4 2020-12-25T21:26:55.677 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-12-26T08:50:10.413 INFO:teuthology.results:Tests finished! gathering results... 2020-12-26T08:50:36.834 INFO:scrape:Found 100 jobs 2020-12-26T08:50:38.569 INFO:scrape:Found 5 distinct failure reasons 2020-12-26T08:50:38.570 INFO:scrape:Failure: "2020-12-25T21:31:12.308971+0000 osd.7 (osd.7) 3 : cluster [WRN] slow request osd_op(client.4414.0:477937 2.1 2:9148bb5c:::rbd_data.10fd30d9cad6.000000000000086e:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3403776~4096 in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e21) initiated 2020-12-25T21:30:42.210246+0000 currently waiting for sub ops" in cluster log 2020-12-26T08:50:38.571 INFO:scrape:1 jobs: ['5719053'] 2020-12-26T08:50:38.571 INFO:scrape:suites: ['clusters/fixed-3', 'conf', 'krbd/rbd/{bluestore-bitmap', 'msgr-failures/many', 'tasks/rbd_workunit_suites_ffsb}'] 2020-12-26T08:50:38.572 INFO:scrape: 2020-12-26T08:50:38.572 INFO:scrape:Failure: "2020-12-25T23:29:43.741135+0000 mon.a (mon.0) 378 : cluster [WRN] Health check failed: 0 slow ops, oldest one blocked for 31 sec, osd.3 has slow ops (SLOW_OPS)" in cluster log 2020-12-26T08:50:38.572 INFO:scrape:1 jobs: ['5719116'] 2020-12-26T08:50:38.573 INFO:scrape:suites: ['clusters/fixed-3', 'conf', 'install/ceph', 'krbd/rbd-nomount/{bluestore-bitmap', 'msgr-failures/many', 'tasks/rbd_concurrent}'] 2020-12-26T08:50:38.573 INFO:scrape: 2020-12-26T08:50:38.574 INFO:scrape:Crash: Command failed on smithi117 with status 4: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeyvra0idg -r -- '-g auto -g blockdev -x clone'" 2020-12-26T08:50:38.574 INFO:scrape:ceph version 16.0.0-8468-g6f7150fb (6f7150fb06f152de9011b145a29f2e44ef3b8947) pacific (dev) 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7f2698892980] 2: gsignal() 3: abort() 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x1a7) [0x55ba7703276f] 5: (PrimaryLogPG::execute_ctx(PrimaryLogPG::OpContext*)+0x1a79) [0x55ba77223ba9] 6: (PrimaryLogPG::do_op(boost::intrusive_ptr&)+0x4006) [0x55ba77227c96] 7: (PrimaryLogPG::do_request(boost::intrusive_ptr&, ThreadPool::TPHandle&)+0xd0a) [0x55ba772331fa] 8: (OSD::dequeue_op(boost::intrusive_ptr, boost::intrusive_ptr, ThreadPool::TPHandle&)+0x17b) [0x55ba770b935b] 9: (ceph::osd::scheduler::PGOpItem::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0x6a) [0x55ba7731caca] 10: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0xcd5) [0x55ba770d72f5] 11: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x4ac) [0x55ba7774d2fc] 12: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0x55ba777505b0] 13: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f26988876db] 14: clone() 2020-12-26T08:50:38.575 INFO:scrape:1 jobs: ['5719087'] 2020-12-26T08:50:38.575 INFO:scrape:suites: ['conf', 'krbd/singleton/{bluestore-bitmap', 'msgr-failures/many', 'tasks/rbd_xfstests}'] 2020-12-26T08:50:38.575 INFO:scrape: 2020-12-26T08:50:38.576 INFO:scrape:Failure: "2020-12-25T21:52:57.314850+0000 osd.3 (osd.3) 4 : cluster [WRN] slow request osd_op(client.4402.0:7799 3.1e 3:7a96495e:::rbd_header.11321301a2a9:head [watch reconnect cookie 139738774960640 gen 5] snapc 0=[] ondisk+write+known_if_redirected e59) initiated 2020-12-25T21:52:26.165175+0000 currently started" in cluster log 2020-12-26T08:50:38.576 INFO:scrape:1 jobs: ['5719130'] 2020-12-26T08:50:38.577 INFO:scrape:suites: ['clusters/3-node', 'conf', 'features/no-object-map', 'krbd/fsx/{ceph/ceph', 'objectstore/bluestore-bitmap', 'randomized-striping-off}', 'striping/default/{msgr-failures/many', 'tasks/fsx-3-client}'] 2020-12-26T08:50:38.577 INFO:scrape: 2020-12-26T08:50:38.578 INFO:scrape:Failure: Command failed on smithi184 with status 4: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude8qz9tygf -r -- '-g auto -g blockdev -x clone'" 2020-12-26T08:50:38.578 INFO:scrape:1 jobs: ['5719036'] 2020-12-26T08:50:38.578 INFO:scrape:suites: ['conf', 'krbd/singleton/{bluestore-bitmap', 'msgr-failures/few', 'tasks/rbd_xfstests}'] 2020-12-26T08:50:38.579 INFO:scrape: 2020-12-26T08:50:38.582 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/ failed: 4 dead: 1 running: 0 waiting: 0 queued: 0 passed: 95 Fail ================================================================= [5719036] krbd/singleton/{bluestore-bitmap conf msgr-failures/few tasks/rbd_xfstests} ----------------------------------------------------------------- time: 09:01:46 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719036/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719036/ sentry: https://sentry.ceph.com/organizations/ceph/?query=64f3264369f2498680737ce65babc1ba Command failed on smithi184 with status 4: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/exclude8qz9tygf -r -- '-g auto -g blockdev -x clone'" [5719053] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_ffsb} ----------------------------------------------------------------- time: 01:38:35 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719053/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719053/ "2020-12-25T21:31:12.308971+0000 osd.7 (osd.7) 3 : cluster [WRN] slow request osd_op(client.4414.0:477937 2.1 2:9148bb5c:::rbd_data.10fd30d9cad6.000000000000086e:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3403776~4096 in=4096b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e21) initiated 2020-12-25T21:30:42.210246+0000 currently waiting for sub ops" in cluster log [5719116] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_concurrent} ----------------------------------------------------------------- time: 03:36:01 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719116/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719116/ "2020-12-25T23:29:43.741135+0000 mon.a (mon.0) 378 : cluster [WRN] Health check failed: 0 slow ops, oldest one blocked for 31 sec, osd.3 has slow ops (SLOW_OPS)" in cluster log [5719130] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} ----------------------------------------------------------------- time: 00:26:05 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719130/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719130/ "2020-12-25T21:52:57.314850+0000 osd.3 (osd.3) 4 : cluster [WRN] slow request osd_op(client.4402.0:7799 3.1e 3:7a96495e:::rbd_header.11321301a2a9:head [watch reconnect cookie 139738774960640 gen 5] snapc 0=[] ondisk+write+known_if_redirected e59) initiated 2020-12-25T21:52:26.165175+0000 currently started" in cluster log Dead ================================================================= [5719087] krbd/singleton/{bluestore-bitmap conf msgr-failures/many tasks/rbd_xfstests} ----------------------------------------------------------------- time: 10:41:38 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719087/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719087/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1ef3d43f146b44ea947a0dea2a0b2c73 Command failed on smithi117 with status 4: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeyvra0idg -r -- '-g auto -g blockdev -x clone'" Pass ================================================================= [5719032] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_deep_flatten} time: 00:17:19 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719032/ [5719033] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:26:47 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719033/ [5719034] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_fio} time: 00:40:49 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719034/ [5719035] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_data_pool} time: 00:29:24 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719035/ [5719037] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/backoff thrashosds-health workloads/rbd_fio} time: 01:27:55 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719037/ [5719038] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/pre-single-major tasks/unmap} time: 00:15:08 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719038/ [5719039] krbd/wac/sysfs/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/stable_writes} time: 00:15:17 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719039/ [5719040] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_exclusive_option} time: 00:19:03 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719040/ [5719041] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:28:11 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719041/ [5719042] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_kernel_untar_build} time: 00:23:31 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719042/ [5719043] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_fallocate} time: 00:18:15 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719043/ [5719044] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:28:53 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719044/ [5719045] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_huge_osdmap} time: 00:18:12 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719045/ [5719046] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_dbench} time: 00:53:29 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719046/ [5719047] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_latest_osdmap_on_map} time: 00:20:57 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719047/ [5719048] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_discard} time: 00:17:22 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719048/ [5719049] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:24:02 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719049/ [5719050] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/mon-thrasher thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:21:13 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719050/ [5719051] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_namespaces} time: 00:22:34 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719051/ [5719052] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:33:08 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719052/ [5719054] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_udev_enumerate} time: 00:36:28 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719054/ [5719055] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_udev_netlink_enobufs} time: 00:25:38 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719055/ [5719056] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:27:52 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719056/ [5719057] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_fsstress} time: 00:17:01 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719057/ [5719058] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_udev_netns} time: 00:17:37 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719058/ [5719059] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_huge_image} time: 00:14:58 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719059/ [5719060] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:25:30 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719060/ [5719061] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/pggrow thrashosds-health workloads/rbd_fio} time: 01:17:29 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719061/ [5719062] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_udev_symlinks} time: 00:18:33 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719062/ [5719063] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_fsstress_ext4} time: 00:17:12 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719063/ [5719064] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 04:59:50 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719064/ [5719065] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_concurrent} time: 00:23:01 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719065/ [5719066] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_huge_tickets} time: 00:19:12 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719066/ [5719067] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:30:17 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719067/ [5719068] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_fsx} time: 00:25:48 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719068/ [5719069] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-off tasks/unmap} time: 00:15:26 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719069/ [5719070] krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/many-resets} time: 00:35:38 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719070/ [5719071] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_image_read} time: 00:17:09 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719071/ [5719072] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_modprobe} time: 00:15:05 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719072/ [5719073] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:28:48 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719073/ [5719074] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/upmap thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:57:54 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719074/ [5719075] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_kernel} time: 00:16:41 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719075/ [5719076] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_iozone} time: 00:20:32 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719076/ [5719077] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_map_snapshot_io} time: 00:16:18 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719077/ [5719078] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:29:22 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719078/ [5719079] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_map_unmap} time: 00:21:32 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719079/ [5719080] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_trivial_sync} time: 00:16:23 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719080/ [5719081] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:33:24 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719081/ [5719082] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_simple_big} time: 00:46:54 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719082/ [5719083] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_msgr_segments} time: 00:15:10 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719083/ [5719084] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:29:26 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719084/ [5719085] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_fio} time: 00:41:17 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719085/ [5719086] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_data_pool} time: 00:34:05 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719086/ [5719088] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 02:17:05 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719088/ [5719089] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_exclusive_option} time: 00:19:06 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719089/ [5719090] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 03:03:07 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719090/ [5719091] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_kernel_untar_build} time: 00:23:20 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719091/ [5719092] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_fallocate} time: 00:18:12 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719092/ [5719093] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:35:26 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719093/ [5719094] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_huge_osdmap} time: 00:18:19 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719094/ [5719095] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_dbench} time: 00:58:25 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719095/ [5719096] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_latest_osdmap_on_map} time: 00:21:40 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719096/ [5719097] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_parent_overlap} time: 00:14:58 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719097/ [5719098] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:24:28 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719098/ [5719099] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/mon-thrasher thrashosds-health workloads/rbd_fio} time: 01:26:18 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719099/ [5719100] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_namespaces} time: 00:19:36 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719100/ [5719101] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:33:46 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719101/ [5719102] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_ffsb} time: 01:23:41 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719102/ [5719103] krbd/unmap/{ceph/ceph clusters/separate-client conf filestore-xfs kernels/single-major-on tasks/unmap} time: 00:15:34 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719103/ [5719104] krbd/wac/wac/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf tasks/wac verify/no-resets} time: 00:28:56 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719104/ [5719105] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_udev_enumerate} time: 02:21:44 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719105/ [5719106] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_udev_netlink_enobufs} time: 00:23:01 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719106/ [5719107] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-3-client} time: 00:29:11 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719107/ [5719108] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_fsstress} time: 00:17:22 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719108/ [5719109] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/krbd_udev_netns} time: 00:18:09 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719109/ [5719110] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_read_only} time: 00:16:37 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719110/ [5719111] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-1-client} time: 00:29:48 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719111/ [5719112] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/pggrow thrashosds-health workloads/rbd_workunit_suites_ffsb} time: 01:43:38 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719112/ [5719113] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/krbd_udev_symlinks} time: 00:18:19 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719113/ [5719114] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_fsstress_ext4} time: 00:17:31 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719114/ [5719115] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-3-client} time: 00:35:58 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719115/ [5719117] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_huge_tickets} time: 00:16:45 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719117/ [5719118] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/default/{msgr-failures/many randomized-striping-off} tasks/fsx-1-client} time: 00:33:38 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719118/ [5719119] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_suites_fsx} time: 00:26:59 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719119/ [5719120] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_image_read} time: 00:17:32 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719120/ [5719121] krbd/basic/{bluestore-bitmap ceph/ceph clusters/fixed-1 conf tasks/krbd_whole_object_zeroout} time: 00:16:09 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719121/ [5719122] krbd/fsx/{ceph/ceph clusters/3-node conf features/no-object-map objectstore/bluestore-bitmap striping/default/{msgr-failures/few randomized-striping-off} tasks/fsx-3-client} time: 00:25:08 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719122/ [5719123] krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf thrashers/upmap thrashosds-health workloads/rbd_fio} time: 01:25:57 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719123/ [5719124] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_kernel} time: 00:16:36 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719124/ [5719125] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/few tasks/rbd_workunit_suites_iozone} time: 00:20:06 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719125/ [5719126] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_map_snapshot_io} time: 00:16:15 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719126/ [5719127] krbd/fsx/{ceph/ceph clusters/3-node conf features/object-map objectstore/filestore-xfs striping/fancy/{msgr-failures/few randomized-striping-on} tasks/fsx-1-client} time: 00:31:53 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719127/ [5719128] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/few tasks/rbd_map_unmap} time: 00:21:13 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719128/ [5719129] krbd/rbd/{bluestore-bitmap clusters/fixed-3 conf msgr-failures/many tasks/rbd_workunit_trivial_sync} time: 00:16:38 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719129/ [5719131] krbd/rbd-nomount/{bluestore-bitmap clusters/fixed-3 conf install/ceph msgr-failures/many tasks/rbd_simple_big} time: 00:48:07 info: http://pulpito.ceph.com/teuthology-2020-12-19_03:20:02-krbd-master-testing-basic-smithi/5719131/