2020-06-23T01:51:09.866 INFO:root:teuthology version: 1.0.0-869528d
2020-06-23T01:51:09.868 INFO:teuthology.results:subset: '8/2999'
2020-06-23T01:51:09.869 INFO:teuthology.results:seed: '1641'
2020-06-23T02:09:07.276 INFO:root:teuthology version: 1.0.0-869528d
2020-06-23T02:09:07.290 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-06-23T04:51:10.352 INFO:teuthology.results:Tests finished! gathering results...
2020-06-23T04:51:10.424 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/
failed:  3
dead:    2
running: 0
waiting: 0
queued:  0
passed:  19


Fail
=================================================================
[5171030]  multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/kclient/{kernel-testing mount ms-die-on-skipped} objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/valgrind}
-----------------------------------------------------------------
time:   02:41:44
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171030/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171030/

    "2020-06-23 03:28:46.481073 mds.c (mds.1) 1 : cluster [WRN] client.4549
    isn't responding to mclientcaps(revoke), ino 0x20000002a50 pending
    pAsLsXsFsxcrwb issued pAsxLsXsxFsxcrwb, sent 60.165885 seconds ago" in
    cluster log

[5171041]  multimds/thrash/{0-supported-random-distro$/{centos_latest} begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
-----------------------------------------------------------------
time:   00:40:54
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171041/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171041/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=c2838acf786b448481581889b020cdcf

    Command failed on smithi031 with status 1: 'sudo rm -rf --
    /home/ubuntu/cephtest/mnt.0/client.0/tmp'

[5171051]  multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/random/{k-testing supported$/{rhel_latest}} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
-----------------------------------------------------------------
time:   01:48:51
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171051/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171051/

    "2020-06-23 02:42:27.270778 mon.b (mon.0) 245 : cluster [WRN] Health check
    failed: 1 MDSs report slow metadata IOs (MDS_SLOW_METADATA_IO)" in cluster
    log



Dead
=================================================================
[5171036]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-bitmap overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171036/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171036/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=d8ce0dc73b6e41a4b09e9a1f5b62fa58

    SSH connection to smithi075 was lost: 'sudo rpm -ivh --oldpackage
    --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm'

[5171052]  multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171052/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171052/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=46c344f9278044af9603b086a13852f0

    SSH connection to smithi003 was lost: 'sudo rpm -ivh --oldpackage
    --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm'



Pass
=================================================================
[5171027] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_ffsb}
time:   00:40:35
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171027/

[5171028] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsstress}
time:   00:26:21
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171028/

[5171032] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:30:36
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171032/

[5171034] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/random/{k-testing supported$/{ubuntu_latest}} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_fsx}
time:   00:27:01
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171034/

[5171035] multimds/basic/{0-supported-random-distro$/{ubuntu_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_pjd}
time:   00:17:53
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171035/

[5171037] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/random/{k-testing supported$/{rhel_latest}} ms-die-on-skipped}} msgr-failures/none objectstore-ec/filestore-xfs overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:19:08
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171037/

[5171038] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_exports}
time:   00:13:11
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171038/

[5171039] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/random/{k-testing supported$/{rhel_latest}} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cephfs_test_snapshots}
time:   00:28:13
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171039/

[5171040] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/kclient/{kernel-testing mount ms-die-on-skipped} objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   00:57:51
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171040/

[5171042] multimds/basic/{0-supported-random-distro$/{rhel_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_kernel_untar_build}
time:   00:57:58
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171042/

[5171043] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/fuse objectstore-ec/bluestore-comp overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_misc}
time:   01:45:06
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171043/

[5171044] multimds/verify/{begin centos_latest clusters/9-mds conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_fsstress validater/lockdep}
time:   00:26:21
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171044/

[5171045] multimds/thrash/{0-supported-random-distro$/{ubuntu_latest} begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:15:59
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171045/

[5171046] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/kclient/{mount overrides/{distro/random/{k-testing supported$/{rhel_latest}} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_norstats}
time:   00:22:02
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171046/

[5171047] multimds/basic/{0-supported-random-distro$/{centos_latest} begin clusters/3-mds conf/{client mds mon osd} inline/no mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} objectstore-ec/filestore-xfs overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_blogbench}
time:   00:29:13
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171047/

[5171048] multimds/verify/{begin centos_latest clusters/3-mds conf/{client mds mon osd} mount/kclient/{kernel-testing mount ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no verify/{frag_enable mon-debug whitelist_health whitelist_wrongly_marked_down}} tasks/cfuse_workunit_suites_dbench validater/valgrind}
time:   00:58:28
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171048/

[5171049] multimds/thrash/{0-supported-random-distro$/{ubuntu_16.04} begin ceph-thrash/default clusters/3-mds-2-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/random/{k-testing supported$/{ubuntu_16.04}} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_fsstress}
time:   00:23:44
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171049/

[5171050] multimds/basic/{0-supported-random-distro$/{ubuntu_16.04} begin clusters/9-mds conf/{client mds mon osd} inline/yes mount/fuse objectstore-ec/bluestore-bitmap overrides/{basic/{frag_enable whitelist_health whitelist_wrongly_marked_down} fuse-default-perm-no} q_check_counter/check_counter tasks/cfuse_workunit_suites_dbench}
time:   00:57:03
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171050/

[5171053] multimds/thrash/{0-supported-random-distro$/{centos_latest} begin ceph-thrash/default clusters/9-mds-3-standby conf/{client mds mon osd} mount/kclient/{mount overrides/{distro/rhel/{k-distro rhel_latest} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{fuse-default-perm-no thrash/{frag_enable whitelist_health whitelist_wrongly_marked_down} thrash_debug} tasks/cfuse_workunit_suites_pjd}
time:   00:18:02
info:   http://pulpito.ceph.com/teuthology-2020-06-22_13:15:03-multimds-nautilus-testing-basic-smithi/5171053/