2020-04-24T13:08:46.563 INFO:root:teuthology version: 1.0.0-876b8741 2020-04-24T13:08:46.565 INFO:teuthology.results:seed: '4112' 2020-04-24T20:12:03.645 INFO:root:teuthology version: 1.0.0-6cf40593 2020-04-24T20:12:03.677 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-04-24T21:50:05.630 INFO:teuthology.results:Tests finished! gathering results... 2020-04-24T21:50:05.796 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/ failed: 13 dead: 0 running: 0 waiting: 0 queued: 0 passed: 83 Fail ================================================================= [4979519] krbd/singleton/{bluestore-bitmap.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_xfstests.yaml} ----------------------------------------------------------------- time: 03:34:29 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979519/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979519/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=d0392d28884f4dee8a589f454b5a83b5 Command failed on smithi188 with status 3: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeAljDIY -r -- '-g auto -g blockdev -x clone'" [4979523] krbd/unmap/{ceph/ceph.yaml clusters/separate-client.yaml conf.yaml filestore-xfs.yaml kernels/pre-single-major.yaml tasks/unmap.yaml} ----------------------------------------------------------------- time: 00:09:48 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979523/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979523/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=e0c6f4296e1b4f10b8eba2c06e58889f Command failed on smithi032 with status 1: 'CEPH_REF=master CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /home/ubuntu/cephtest/virtualenv/bin/cram -v -- /home/ubuntu/cephtest/archive/cram.client.0/*.t' [4979545] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/mon-thrasher.yaml thrashosds-health.yaml workloads/rbd_workunit_suites_ffsb.yaml} ----------------------------------------------------------------- time: 01:05:25 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979545/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979545/ "2020-04-24T14:04:20.286571+0000 mon.a (mon.1) 34 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log [4979551] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_ffsb.yaml} ----------------------------------------------------------------- time: 01:15:50 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979551/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979551/ "2020-04-24T15:22:51.870793+0000 osd.5 (osd.5) 3 : cluster [WRN] slow request osd_op(client.4450.0:708985 2.3 2:d97f54a9:::rbd_data.116793f55bd8.000000000000060a:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2961408~8192 in=8192b] snapc 0=[] RETRY=4 ondisk+retry+write+known_if_redirected e22) initiated 2020-04-24T15:22:21.699322+0000 currently waiting for sub ops" in cluster log [4979569] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_concurrent.yaml} ----------------------------------------------------------------- time: 00:16:38 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979569/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979569/ Found coredumps on ubuntu@smithi201.front.sepia.ceph.com [4979577] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} ----------------------------------------------------------------- time: 00:30:32 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979577/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979577/ "2020-04-24T15:20:50.968399+0000 osd.6 (osd.6) 3 : cluster [WRN] slow request osd_op(client.4446.0:54841 3.10 3:0ae1d6da:::rbd_data.115ed700d174.000000000000002b:head [delete] snapc 0=[] ondisk+write+known_if_redirected e117) initiated 2020-04-24T15:20:20.782417+0000 currently started" in cluster log [4979583] krbd/unmap/{ceph/ceph.yaml clusters/separate-client.yaml conf.yaml filestore-xfs.yaml kernels/single-major-off.yaml tasks/unmap.yaml} ----------------------------------------------------------------- time: 00:10:11 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979583/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979583/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=4eb89d95cf6945efad417dd9220b31e9 Command failed on smithi149 with status 1: 'CEPH_REF=master CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /home/ubuntu/cephtest/virtualenv/bin/cram -v -- /home/ubuntu/cephtest/archive/cram.client.0/*.t' [4979600] krbd/singleton/{bluestore-bitmap.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_xfstests.yaml} ----------------------------------------------------------------- time: 04:04:50 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979600/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979600/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=fa2bc1af4f1c4de186a4f68a47e9a800 Command failed on smithi130 with status 3: "/usr/bin/sudo TESTDIR=/home/ubuntu/cephtest adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/bash /home/ubuntu/cephtest/run_xfstests.sh -c 1 -f ext4 -t /dev/rbd0 -s /dev/rbd1 -x /tmp/excludeEI8Cmu -r -- '-g auto -g blockdev -x clone'" [4979611] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/mon-thrasher.yaml thrashosds-health.yaml workloads/rbd_fio.yaml} ----------------------------------------------------------------- time: 00:51:58 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979611/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979611/ "2020-04-24T17:13:31.118585+0000 mon.a (mon.1) 95 : cluster [WRN] Health check failed: 1/3 mons down, quorum a,c (MON_DOWN)" in cluster log [4979616] krbd/unmap/{ceph/ceph.yaml clusters/separate-client.yaml conf.yaml filestore-xfs.yaml kernels/single-major-on.yaml tasks/unmap.yaml} ----------------------------------------------------------------- time: 00:10:02 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979616/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979616/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=97ab2e628f4c4315b4d64bcf042b118f Command failed on smithi205 with status 1: 'CEPH_REF=master CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /home/ubuntu/cephtest/virtualenv/bin/cram -v -- /home/ubuntu/cephtest/archive/cram.client.0/*.t' [4979635] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/upmap.yaml thrashosds-health.yaml workloads/rbd_fio.yaml} ----------------------------------------------------------------- time: 00:03:39 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979635/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979635/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=9ee59c401f49476ab7363812f1c0a46a 502 Server Error: Bad Gateway for url: https://3.chacra.ceph.com/repos/ceph /octopus/b43a03dce178d45a2921f2d3b58e790cfabdede6/ubuntu/bionic/flavors/def ault/repo [4979640] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_trivial_sync.yaml} ----------------------------------------------------------------- time: 00:03:32 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979640/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979640/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=a132ccc7b93347e897a775a9b0eac66e 502 Server Error: Bad Gateway for url: https://3.chacra.ceph.com/repos/ceph /octopus/b43a03dce178d45a2921f2d3b58e790cfabdede6/ubuntu/bionic/flavors/def ault/repo [4979641] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} ----------------------------------------------------------------- time: 00:03:28 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979641/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979641/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=95d69650826e41ae81805ee8af490390 502 Server Error: Bad Gateway for url: https://3.chacra.ceph.com/repos/ceph /octopus/b43a03dce178d45a2921f2d3b58e790cfabdede6/ubuntu/bionic/flavors/def ault/repo Pass ================================================================= [4979511] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_deep_flatten.yaml} time: 00:12:54 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979511/ [4979513] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:25:57 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979513/ [4979515] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_fio.yaml} time: 00:32:43 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979515/ [4979517] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_data_pool.yaml} time: 00:22:03 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979517/ [4979521] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/backoff.yaml thrashosds-health.yaml workloads/rbd_fio.yaml} time: 00:51:51 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979521/ [4979525] krbd/wac/sysfs/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/stable_pages_required.yaml} time: 00:09:55 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979525/ [4979527] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_exclusive_option.yaml} time: 00:14:07 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979527/ [4979529] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/filestore-xfs.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-3-client.yaml} time: 00:33:05 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979529/ [4979531] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_kernel_untar_build.yaml} time: 00:17:16 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979531/ [4979533] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_fallocate.yaml} time: 00:12:28 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979533/ [4979535] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:30:25 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979535/ [4979537] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_huge_osdmap.yaml} time: 00:13:49 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979537/ [4979539] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_dbench.yaml} time: 00:41:58 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979539/ [4979541] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_discard.yaml} time: 00:11:20 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979541/ [4979543] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:28:01 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979543/ [4979547] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_latest_osdmap_on_map.yaml} time: 00:15:15 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979547/ [4979549] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/bluestore-bitmap.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-1-client.yaml} time: 00:37:31 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979549/ [4979553] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_namespaces.yaml} time: 00:15:29 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979553/ [4979555] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_udev_enumerate.yaml} time: 00:25:51 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979555/ [4979557] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:28:38 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979557/ [4979559] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_fsstress.yaml} time: 00:11:34 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979559/ [4979561] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_udev_netlink_enobufs.yaml} time: 00:16:30 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979561/ [4979563] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_huge_image.yaml} time: 00:10:26 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979563/ [4979565] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:26:21 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979565/ [4979567] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/rbd_fio.yaml} time: 00:55:42 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979567/ [4979571] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_fsstress_ext4.yaml} time: 00:13:07 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979571/ [4979573] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/bluestore-bitmap.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-3-client.yaml} time: 00:30:45 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979573/ [4979575] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_huge_tickets.yaml} time: 00:11:46 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979575/ [4979579] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_fsx.yaml} time: 00:17:44 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979579/ [4979581] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_image_read.yaml} time: 00:11:22 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979581/ [4979584] krbd/wac/wac/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml tasks/wac.yaml verify/many-resets.yaml} time: 00:22:51 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979584/ [4979585] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_kernel.yaml} time: 00:11:03 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979585/ [4979586] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_modprobe.yaml} time: 00:09:40 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979586/ [4979587] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:28:26 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979587/ [4979588] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/upmap.yaml thrashosds-health.yaml workloads/rbd_workunit_suites_ffsb.yaml} time: 01:01:44 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979588/ [4979589] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_iozone.yaml} time: 00:15:00 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979589/ [4979590] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_map_snapshot_io.yaml} time: 00:11:02 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979590/ [4979591] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/filestore-xfs.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-1-client.yaml} time: 00:43:01 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979591/ [4979592] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_map_unmap.yaml} time: 00:15:50 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979592/ [4979593] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_trivial_sync.yaml} time: 00:11:01 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979593/ [4979594] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:30:28 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979594/ [4979595] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_simple_big.yaml} time: 00:25:14 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979595/ [4979596] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_msgr_segments.yaml} time: 00:09:51 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979596/ [4979597] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:29:55 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979597/ [4979598] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_fio.yaml} time: 00:36:05 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979598/ [4979599] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_data_pool.yaml} time: 00:25:41 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979599/ [4979601] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/backoff.yaml thrashosds-health.yaml workloads/rbd_workunit_suites_ffsb.yaml} time: 01:03:41 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979601/ [4979602] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_exclusive_option.yaml} time: 00:13:11 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979602/ [4979603] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/filestore-xfs.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-3-client.yaml} time: 00:34:38 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979603/ [4979604] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_kernel_untar_build.yaml} time: 00:17:02 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979604/ [4979605] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_fallocate.yaml} time: 00:12:38 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979605/ [4979606] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:31:51 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979606/ [4979607] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_huge_osdmap.yaml} time: 00:12:04 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979607/ [4979608] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_dbench.yaml} time: 00:40:49 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979608/ [4979609] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_parent_overlap.yaml} time: 00:10:03 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979609/ [4979610] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:25:37 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979610/ [4979612] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_latest_osdmap_on_map.yaml} time: 00:15:47 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979612/ [4979613] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/bluestore-bitmap.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-1-client.yaml} time: 00:35:36 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979613/ [4979614] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_ffsb.yaml} time: 01:00:25 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979614/ [4979615] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_namespaces.yaml} time: 00:13:14 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979615/ [4979617] krbd/wac/wac/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml tasks/wac.yaml verify/no-resets.yaml} time: 00:23:01 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979617/ [4979618] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/krbd_udev_enumerate.yaml} time: 00:38:37 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979618/ [4979619] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:28:12 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979619/ [4979620] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_fsstress.yaml} time: 00:11:36 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979620/ [4979621] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/krbd_udev_netlink_enobufs.yaml} time: 00:16:01 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979621/ [4979622] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_read_only.yaml} time: 00:10:06 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979622/ [4979623] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:27:42 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979623/ [4979624] krbd/thrash/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-3.yaml conf.yaml thrashers/pggrow.yaml thrashosds-health.yaml workloads/rbd_workunit_suites_ffsb.yaml} time: 01:00:17 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979624/ [4979625] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_concurrent.yaml} time: 00:17:37 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979625/ [4979626] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_fsstress_ext4.yaml} time: 00:11:10 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979626/ [4979627] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/bluestore-bitmap.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-3-client.yaml} time: 01:04:34 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979627/ [4979628] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_huge_tickets.yaml} time: 00:11:02 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979628/ [4979629] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/filestore-xfs.yaml striping/default/{msgr-failures/many.yaml randomized-striping-off.yaml} tasks/fsx-1-client.yaml} time: 00:34:08 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979629/ [4979630] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/many.yaml tasks/rbd_workunit_suites_fsx.yaml} time: 00:19:27 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979630/ [4979631] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_image_read.yaml} time: 00:11:43 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979631/ [4979632] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_kernel.yaml} time: 00:11:02 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979632/ [4979633] krbd/basic/{bluestore-bitmap.yaml ceph/ceph.yaml clusters/fixed-1.yaml conf.yaml tasks/krbd_whole_object_zeroout.yaml} time: 00:10:44 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979633/ [4979634] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/no-object-map.yaml objectstore/bluestore-bitmap.yaml striping/default/{msgr-failures/few.yaml randomized-striping-off.yaml} tasks/fsx-3-client.yaml} time: 00:26:13 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979634/ [4979636] krbd/rbd/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml msgr-failures/few.yaml tasks/rbd_workunit_suites_iozone.yaml} time: 00:13:42 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979636/ [4979637] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_map_snapshot_io.yaml} time: 00:11:04 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979637/ [4979638] krbd/fsx/{ceph/ceph.yaml clusters/3-node.yaml conf.yaml features/object-map.yaml objectstore/filestore-xfs.yaml striping/fancy/{msgr-failures/few.yaml randomized-striping-on.yaml} tasks/fsx-1-client.yaml} time: 00:33:48 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979638/ [4979639] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/few.yaml tasks/rbd_map_unmap.yaml} time: 00:15:57 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979639/ [4979642] krbd/rbd-nomount/{bluestore-bitmap.yaml clusters/fixed-3.yaml conf.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_simple_big.yaml} time: 00:27:31 info: http://pulpito.ceph.com/teuthology-2020-04-24_05:15:02-krbd-octopus-testing-basic-smithi/4979642/ 2020-04-24T21:50:05.900 INFO:teuthology.results:starting coverage generation 2020-04-24T21:50:05.904 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 35, in main int(args['--timeout']), args['--dry-run']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 84, in results generate_coverage(archive_dir, name) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 107, in generate_coverage archive_dir, File "/usr/lib/python3.6/subprocess.py", line 729, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage': '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage'