2020-06-25T07:05:14.757 INFO:root:teuthology version: 1.0.0-37e120c 2020-06-25T07:05:14.759 INFO:teuthology.results:subset: '10/9999' 2020-06-25T07:05:14.759 INFO:teuthology.results:seed: '8862' 2020-06-25T07:13:19.748 INFO:root:teuthology version: 1.0.0-37e120c 2020-06-25T07:13:19.756 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-06-25T11:47:25.567 INFO:teuthology.results:Tests finished! gathering results... 2020-06-25T11:47:25.639 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/ failed: 5 dead: 0 running: 0 waiting: 0 queued: 0 passed: 19 Fail ================================================================= [5175600] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind} ----------------------------------------------------------------- time: 04:11:09 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175600/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175600/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=2d2ba5a763674ce58447c9cb089f3f09 [5175608] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind} ----------------------------------------------------------------- time: 03:37:28 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175608/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175608/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=c7fe4d60ce994e97837fcaa4d14be769 [5175609] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} ----------------------------------------------------------------- time: 00:02:58 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175609/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175609/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=1c846411d1a4457fbba912d6b27112b0 Command failed on smithi129 with status 1: 'sudo yum -y install ceph- radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph- mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph- mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd- mirror rbd-nbd python3-cephfs bison flex elfutils-libelf-devel openssl- devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio- devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs- devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel' [5175614] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench} ----------------------------------------------------------------- time: 03:23:57 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175614/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175614/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=6e4fea1e3ad6434db6524eb0d7f2af82 Command failed (workunit test suites/blogbench.sh) on smithi012 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5c9eee5a7ad1bb8b161f33b4560d5dfc4a000b6d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/blogbench.sh' [5175616] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind} ----------------------------------------------------------------- time: 03:25:35 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175616/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175616/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=f7263945613b403f95525d3aa996fe78 Pass ================================================================= [5175599] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx} time: 00:23:32 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175599/ [5175601] multimds/thrash/{0-supported-random-distro$/{rhel_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} time: 00:24:28 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175601/ [5175602] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd} time: 00:22:37 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175602/ [5175603] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports} time: 00:17:50 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175603/ [5175604] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep} time: 01:02:13 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175604/ [5175605] multimds/thrash/{0-supported-random-distro$/{rhel_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} time: 00:23:21 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175605/ [5175606] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots} time: 00:24:07 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175606/ [5175607] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build} time: 00:56:37 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175607/ [5175610] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc} time: 00:56:18 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175610/ [5175611] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats} time: 00:21:00 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175611/ [5175612] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep} time: 00:56:06 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175612/ [5175613] multimds/thrash/{0-supported-random-distro$/{centos_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} time: 00:19:15 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175613/ [5175615] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench} time: 00:58:06 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175615/ [5175617] multimds/thrash/{0-supported-random-distro$/{centos_latest} begin ceph-thrash/mds clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd} time: 00:17:01 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175617/ [5175618] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb} time: 01:26:07 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175618/ [5175619] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress} time: 00:23:59 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175619/ [5175620] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/lockdep} time: 01:05:51 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175620/ [5175621] multimds/thrash/{0-supported-random-distro$/{centos_latest} begin ceph-thrash/mon clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress} time: 00:20:51 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175621/ [5175622] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/testing/{flavor/centos_latest k-testing} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx} time: 00:23:39 info: http://pulpito.ceph.com/teuthology-2020-06-24_13:15:04-multimds-octopus-testing-basic-smithi/5175622/