2020-05-05T01:19:10.669 INFO:root:teuthology version: 1.0.0-58684fc5 2020-05-05T01:19:10.671 INFO:teuthology.results:seed: '4144' 2020-05-05T04:31:46.553 INFO:root:teuthology version: 1.0.0-58684fc5 2020-05-05T04:31:46.582 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-05-05T07:20:50.697 INFO:teuthology.results:Tests finished! gathering results... 2020-05-05T07:20:50.866 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/ failed: 5 dead: 0 running: 0 waiting: 0 queued: 0 passed: 91 Fail ================================================================= [5019461] krbd/singleton/{bluestore-bitmap.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_xfstests.yaml} ----------------------------------------------------------------- time: 03:45:26 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019461/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019461/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=38c7c62c9f434043b87ed55f20d48351 Command failed on smithi190 with status 3: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludekebe7dpb -r -- '-g auto -g blockdev -x clone'" [5019474] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/mon-thrasher.yaml thrashosds-health.yaml workloads/rbd_workunit_suites_ffsb.yaml} ----------------------------------------------------------------- time: 01:14:09 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019474/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019474/ "2020-05-05T01:59:06.093916+0000 mon.a (mon.1) 34 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log [5019510] krbd/singleton/{bluestore-bitmap.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_xfstests.yaml} ----------------------------------------------------------------- time: 04:25:53 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019510/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019510/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=11a73d5146704194b85582ae233b3b23 Command failed on smithi201 with status 3: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludehem17ycb -r -- '-g auto -g blockdev -x clone'" [5019521] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/mon-thrasher.yaml thrashosds-health.yaml workloads/rbd_fio.yaml} ----------------------------------------------------------------- time: 00:59:00 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019521/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019521/ "2020-05-05T02:42:32.236101+0000 mon.a (mon.0) 129 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log [5019535] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_concurrent.yaml} ----------------------------------------------------------------- time: 01:29:28 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019535/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019535/ "2020-05-05T02:58:33.738243+0000 osd.1 (osd.1) 3 : cluster [WRN] slow request osd_op(client.10153.0:298487 2.6 2:78101996:::rbd_header.270dee17184a:head [watch reconnect cookie 18446462598732840974 gen 5526] snapc 0=[] ondisk+write+known_if_redirected e21) initiated 2020-05-05T02:58:03.653397+0000 currently started" in cluster log Pass ================================================================= [5019457] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_deep_flatten.yaml} time: 00:21:41 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019457/ [5019458] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:25:25 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019458/ [5019459] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_fio.yaml} time: 00:42:46 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019459/ [5019460] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_data_pool.yaml} time: 00:34:46 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019460/ [5019462] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/backoff.yaml thrashosds-health.yaml workloads/rbd_fio.yaml} time: 01:02:21 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019462/ [5019463] krbd/unmap/{ceph/ceph.yaml clusters/separate-client.yaml conf.yaml filestore-xfs.yaml kernels/pre-single-major.yaml tasks/unmap.yaml} time: 00:20:40 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019463/ [5019464] krbd/wac/sysfs/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/stable_pages_required.yaml} time: 00:19:51 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019464/ [5019465] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_exclusive_option.yaml} time: 00:24:43 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019465/ [5019466] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/filestore-xfs.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-3-client.yaml} time: 00:38:45 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019466/ [5019467] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_kernel_untar_build.yaml} time: 00:31:40 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019467/ [5019468] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_fallocate.yaml} time: 00:21:18 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019468/ [5019469] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:41:35 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019469/ [5019470] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_huge_osdmap.yaml} time: 00:23:11 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019470/ [5019471] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_dbench.yaml} time: 00:52:21 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019471/ [5019472] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_discard.yaml} time: 00:10:29 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019472/ [5019473] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:24:24 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019473/ [5019475] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_latest_osdmap_on_map.yaml} time: 00:25:16 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019475/ [5019476] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/bluestore-bitmap.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-1-client.yaml} time: 00:41:41 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019476/ [5019477] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_ffsb.yaml} time: 01:13:44 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019477/ [5019478] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_namespaces.yaml} time: 00:13:49 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019478/ [5019479] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_udev_enumerate.yaml} time: 00:36:25 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019479/ [5019480] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:42:06 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019480/ [5019481] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_fsstress.yaml} time: 00:22:15 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019481/ [5019482] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_udev_netlink_enobufs.yaml} time: 00:26:52 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019482/ [5019483] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_huge_image.yaml} time: 00:20:44 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019483/ [5019484] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:35:37 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019484/ [5019485] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/rbd_fio.yaml} time: 00:55:37 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019485/ [5019486] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_concurrent.yaml} time: 00:27:15 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019486/ [5019487] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_fsstress_ext4.yaml} time: 00:22:38 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019487/ [5019488] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/bluestore-bitmap.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-3-client.yaml} time: 00:40:31 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019488/ [5019489] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_huge_tickets.yaml} time: 00:21:05 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019489/ [5019490] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:39:55 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019490/ [5019491] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_fsx.yaml} time: 00:24:11 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019491/ [5019492] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_image_read.yaml} time: 00:11:02 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019492/ [5019493] krbd/unmap/{ceph/ceph.yaml clusters/separate-client.yaml conf.yaml filestore-xfs.yaml kernels/single-major-off.yaml tasks/unmap.yaml} time: 00:20:48 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019493/ [5019494] krbd/wac/wac/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml tasks/wac.yaml verify/many-resets.yaml} time: 00:31:27 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019494/ [5019495] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_kernel.yaml} time: 00:20:05 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019495/ [5019496] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_modprobe.yaml} time: 00:09:04 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019496/ [5019497] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:34:22 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019497/ [5019498] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/upmap.yaml thrashosds-health.yaml workloads/rbd_workunit_suites_ffsb.yaml} time: 01:19:32 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019498/ [5019499] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_iozone.yaml} time: 00:24:51 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019499/ [5019500] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_map_snapshot_io.yaml} time: 00:20:37 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019500/ [5019501] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/filestore-xfs.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-1-client.yaml} time: 00:42:29 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019501/ [5019502] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_map_unmap.yaml} time: 00:24:02 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019502/ [5019503] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_trivial_sync.yaml} time: 00:20:07 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019503/ [5019504] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:36:55 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019504/ [5019505] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_simple_big.yaml} time: 00:33:21 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019505/ [5019506] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_msgr_segments.yaml} time: 00:19:57 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019506/ [5019507] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:39:12 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019507/ [5019508] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_fio.yaml} time: 00:38:02 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019508/ [5019509] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_data_pool.yaml} time: 00:33:37 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019509/ [5019511] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/backoff.yaml thrashosds-health.yaml workloads/rbd_workunit_suites_ffsb.yaml} time: 01:24:49 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019511/ [5019512] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_exclusive_option.yaml} time: 00:23:17 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019512/ [5019513] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/filestore-xfs.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-3-client.yaml} time: 00:43:57 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019513/ [5019514] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_kernel_untar_build.yaml} time: 00:27:04 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019514/ [5019515] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_fallocate.yaml} time: 00:21:28 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019515/ [5019516] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:44:10 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019516/ [5019517] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_huge_osdmap.yaml} time: 00:11:48 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019517/ [5019518] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_dbench.yaml} time: 00:49:01 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019518/ [5019519] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_parent_overlap.yaml} time: 00:19:38 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019519/ [5019520] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:24:10 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019520/ [5019522] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_latest_osdmap_on_map.yaml} time: 00:23:16 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019522/ [5019523] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/bluestore-bitmap.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-1-client.yaml} time: 00:41:03 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019523/ [5019524] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_ffsb.yaml} time: 01:10:53 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019524/ [5019525] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_namespaces.yaml} time: 00:13:05 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019525/ [5019526] krbd/unmap/{ceph/ceph.yaml clusters/separate-client.yaml conf.yaml filestore-xfs.yaml kernels/single-major-on.yaml tasks/unmap.yaml} time: 00:19:17 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019526/ [5019527] krbd/wac/wac/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml tasks/wac.yaml verify/no-resets.yaml} time: 00:31:27 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019527/ [5019528] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_udev_enumerate.yaml} time: 00:34:45 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019528/ [5019529] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:34:45 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019529/ [5019530] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_fsstress.yaml} time: 00:12:00 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019530/ [5019531] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_udev_netlink_enobufs.yaml} time: 00:25:31 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019531/ [5019532] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_read_only.yaml} time: 00:20:02 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019532/ [5019533] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:41:45 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019533/ [5019534] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/rbd_workunit_suites_ffsb.yaml} time: 01:09:02 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019534/ [5019536] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_fsstress_ext4.yaml} time: 00:20:20 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019536/ [5019537] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/bluestore-bitmap.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-3-client.yaml} time: 00:38:16 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019537/ [5019538] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_huge_tickets.yaml} time: 00:20:49 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019538/ [5019539] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:44:55 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019539/ [5019540] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_fsx.yaml} time: 00:27:30 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019540/ [5019541] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_image_read.yaml} time: 00:11:07 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019541/ [5019542] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_kernel.yaml} time: 00:10:21 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019542/ [5019543] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_whole_object_zeroout.yaml} time: 00:10:06 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019543/ [5019544] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:24:26 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019544/ [5019545] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/upmap.yaml thrashosds-health.yaml workloads/rbd_fio.yaml} time: 00:49:19 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019545/ [5019546] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_iozone.yaml} time: 00:12:59 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019546/ [5019547] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_map_snapshot_io.yaml} time: 00:10:17 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019547/ [5019548] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/filestore-xfs.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-1-client.yaml} time: 00:30:11 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019548/ [5019549] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_map_unmap.yaml} time: 00:15:17 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019549/ [5019550] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_trivial_sync.yaml} time: 00:11:10 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019550/ [5019551] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:28:00 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019551/ [5019552] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_simple_big.yaml} time: 00:29:11 info: http://pulpito.ceph.com/teuthology-2020-05-04_03:20:02-krbd-master-testing-basic-smithi/5019552/ 2020-05-05T07:20:50.971 INFO:teuthology.results:starting coverage generation 2020-05-05T07:20:50.977 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 35, in main int(args['--timeout']), args['--dry-run']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 84, in results generate_coverage(archive_dir, name) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 107, in generate_coverage archive_dir, File "/usr/lib/python3.6/subprocess.py", line 729, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage': '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage'