2020-04-12T15:36:53.680 INFO:root:teuthology version: 1.0.0-df3de94
2020-04-12T15:36:53.681 INFO:teuthology.results:subset: '18/9999'
2020-04-12T15:36:53.681 INFO:teuthology.results:seed: '9915'
2020-04-12T15:45:06.883 INFO:root:teuthology version: 1.0.0-df3de94
2020-04-12T15:45:07.173 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-04-12T17:47:09.263 INFO:teuthology.results:Tests finished! gathering results...
2020-04-12T17:47:09.331 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/
failed:  4
dead:    1
running: 0
waiting: 0
queued:  0
passed:  19


Fail
=================================================================
[4942671]  multimds/verify/{begin.yaml centos_latest.yaml clusters/9-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{fuse-default-perm-no.yaml verify/{frag_enable.yaml mon-debug.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml}} tasks/cfuse_workunit_suites_dbench.yaml validater/valgrind.yaml}
-----------------------------------------------------------------
time:   01:03:25
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942671/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942671/

    reached maximum tries (50) after waiting for 300 seconds

[4942678]  multimds/basic/{0-supported-random-distro$/{centos_8.yaml} begin.yaml clusters/9-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/kclient/{mount.yaml overrides/{distro/stock/{k-stock.yaml rhel_8.yaml} ms-die-on-skipped.yaml}} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{basic/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} fuse-default-perm-no.yaml} q_check_counter/check_counter.yaml tasks/cephfs_test_snapshots.yaml}
-----------------------------------------------------------------
time:   00:32:28
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942678/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942678/

    [Errno None] Unable to connect to port 22 on 172.21.15.173

[4942673]  multimds/basic/{0-supported-random-distro$/{rhel_8.yaml} begin.yaml clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/kclient/{mount.yaml overrides/{distro/testing/{flavor/centos_latest.yaml k-testing.yaml} ms-die-on-skipped.yaml}} objectstore-ec/bluestore-ec-root.yaml overrides/{basic/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} fuse-default-perm-no.yaml} q_check_counter/check_counter.yaml tasks/cfuse_workunit_suites_fsx.yaml}
-----------------------------------------------------------------
time:   00:24:56
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942673/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942673/

    Command failed on smithi183 with status 1: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster
    ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok perf dump'

[4942687]  multimds/verify/{begin.yaml centos_latest.yaml clusters/9-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{fuse-default-perm-no.yaml verify/{frag_enable.yaml mon-debug.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml}} tasks/cfuse_workunit_suites_dbench.yaml validater/valgrind.yaml}
-----------------------------------------------------------------
time:   00:57:17
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942687/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942687/

    "2020-04-12T16:49:04.645197+0000 mon.a (mon.0) 112 : cluster [WRN] Health
    check failed: Reduced data availability: 1 pg inactive, 1 pg peering
    (PG_AVAILABILITY)" in cluster log



Dead
=================================================================
[4942683]  multimds/verify/{begin.yaml centos_latest.yaml clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/kclient/{k-testing.yaml mount.yaml ms-die-on-skipped.yaml} objectstore-ec/filestore-xfs.yaml overrides/{fuse-default-perm-no.yaml verify/{frag_enable.yaml mon-debug.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml}} tasks/cfuse_workunit_suites_fsstress.yaml validater/lockdep.yaml}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942683/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942683/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=157fccd418fb4f24858828cf6561eab4

    reached maximum tries (60) after waiting for 900 seconds



Pass
=================================================================
[4942679] multimds/verify/{begin.yaml centos_latest.yaml clusters/9-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{fuse-default-perm-no.yaml verify/{frag_enable.yaml mon-debug.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml}} tasks/cfuse_workunit_suites_dbench.yaml validater/valgrind.yaml}
time:   00:59:25
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942679/

[4942688] multimds/thrash/{0-supported-random-distro$/{centos_8.yaml} begin.yaml ceph-thrash/mon.yaml clusters/3-mds-2-standby.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/kclient/{mount.yaml overrides/{distro/stock/{k-stock.yaml rhel_8.yaml} ms-die-on-skipped.yaml}} msgr-failures/osd-mds-delay.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{fuse-default-perm-no.yaml thrash/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrash_debug.yaml} tasks/cfuse_workunit_suites_fsstress.yaml}
time:   00:23:48
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942688/

[4942670] multimds/basic/{0-supported-random-distro$/{ubuntu_latest.yaml} begin.yaml clusters/9-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/kclient/{mount.yaml overrides/{distro/stock/{k-stock.yaml rhel_8.yaml} ms-die-on-skipped.yaml}} objectstore-ec/bluestore-comp.yaml overrides/{basic/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} fuse-default-perm-no.yaml} q_check_counter/check_counter.yaml tasks/cfuse_workunit_suites_fsstress.yaml}
time:   00:22:26
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942670/

[4942672] multimds/thrash/{0-supported-random-distro$/{centos_8.yaml} begin.yaml ceph-thrash/mon.yaml clusters/3-mds-2-standby.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/kclient/{mount.yaml overrides/{distro/stock/{k-stock.yaml rhel_8.yaml} ms-die-on-skipped.yaml}} msgr-failures/osd-mds-delay.yaml objectstore-ec/filestore-xfs.yaml overrides/{fuse-default-perm-no.yaml thrash/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrash_debug.yaml} tasks/cfuse_workunit_suites_fsstress.yaml}
time:   00:24:19
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942672/

[4942675] multimds/verify/{begin.yaml centos_latest.yaml clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/kclient/{k-testing.yaml mount.yaml ms-die-on-skipped.yaml} objectstore-ec/bluestore-comp.yaml overrides/{fuse-default-perm-no.yaml verify/{frag_enable.yaml mon-debug.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml}} tasks/cfuse_workunit_suites_fsstress.yaml validater/lockdep.yaml}
time:   00:16:44
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942675/

[4942674] multimds/basic/{0-supported-random-distro$/{ubuntu_latest.yaml} begin.yaml clusters/9-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/kclient/{mount.yaml overrides/{distro/testing/{flavor/ubuntu_latest.yaml k-testing.yaml} ms-die-on-skipped.yaml}} objectstore-ec/filestore-xfs.yaml overrides/{basic/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} fuse-default-perm-no.yaml} q_check_counter/check_counter.yaml tasks/cfuse_workunit_suites_pjd.yaml}
time:   00:13:19
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942674/

[4942677] multimds/basic/{0-supported-random-distro$/{centos_8.yaml} begin.yaml clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{basic/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} fuse-default-perm-no.yaml} q_check_counter/check_counter.yaml tasks/cephfs_test_exports.yaml}
time:   00:16:10
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942677/

[4942676] multimds/thrash/{0-supported-random-distro$/{rhel_8.yaml} begin.yaml ceph-thrash/mds.yaml clusters/9-mds-3-standby.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/kclient/{mount.yaml overrides/{distro/testing/{flavor/centos_latest.yaml k-testing.yaml} ms-die-on-skipped.yaml}} msgr-failures/none.yaml objectstore-ec/bluestore-bitmap.yaml overrides/{fuse-default-perm-no.yaml thrash/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrash_debug.yaml} tasks/cfuse_workunit_suites_pjd.yaml}
time:   00:13:18
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942676/

[4942666] multimds/basic/{0-supported-random-distro$/{ubuntu_latest.yaml} begin.yaml clusters/9-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/kclient/{mount.yaml overrides/{distro/testing/{flavor/ubuntu_latest.yaml k-testing.yaml} ms-die-on-skipped.yaml}} objectstore-ec/bluestore-bitmap.yaml overrides/{basic/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} fuse-default-perm-no.yaml} q_check_counter/check_counter.yaml tasks/cfuse_workunit_suites_dbench.yaml}
time:   00:54:09
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942666/

[4942667] multimds/thrash/{0-supported-random-distro$/{centos_8.yaml} begin.yaml ceph-thrash/mds.yaml clusters/9-mds-3-standby.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml msgr-failures/none.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{fuse-default-perm-no.yaml thrash/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrash_debug.yaml} tasks/cfuse_workunit_suites_pjd.yaml}
time:   00:15:50
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942667/

[4942665] multimds/basic/{0-supported-random-distro$/{rhel_8.yaml} begin.yaml clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/kclient/{mount.yaml overrides/{distro/testing/{flavor/centos_latest.yaml k-testing.yaml} ms-die-on-skipped.yaml}} objectstore-ec/filestore-xfs.yaml overrides/{basic/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} fuse-default-perm-no.yaml} q_check_counter/check_counter.yaml tasks/cfuse_workunit_suites_blogbench.yaml}
time:   00:28:38
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942665/

[4942686] multimds/basic/{0-supported-random-distro$/{centos_8.yaml} begin.yaml clusters/9-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/kclient/{mount.yaml overrides/{distro/stock/{k-stock.yaml rhel_8.yaml} ms-die-on-skipped.yaml}} objectstore-ec/bluestore-bitmap.yaml overrides/{basic/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} fuse-default-perm-no.yaml} q_check_counter/check_counter.yaml tasks/cfuse_workunit_suites_blogbench.yaml}
time:   00:42:13
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942686/

[4942680] multimds/thrash/{0-supported-random-distro$/{centos_8.yaml} begin.yaml ceph-thrash/mon.yaml clusters/3-mds-2-standby.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/kclient/{mount.yaml overrides/{distro/testing/{flavor/ubuntu_latest.yaml k-testing.yaml} ms-die-on-skipped.yaml}} msgr-failures/osd-mds-delay.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{fuse-default-perm-no.yaml thrash/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrash_debug.yaml} tasks/cfuse_workunit_suites_fsstress.yaml}
time:   00:21:36
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942680/

[4942681] multimds/basic/{0-supported-random-distro$/{centos_8.yaml} begin.yaml clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/kclient/{mount.yaml overrides/{distro/testing/{flavor/centos_latest.yaml k-testing.yaml} ms-die-on-skipped.yaml}} objectstore-ec/bluestore-comp.yaml overrides/{basic/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} fuse-default-perm-no.yaml} q_check_counter/check_counter.yaml tasks/cfuse_workunit_kernel_untar_build.yaml}
time:   00:49:30
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942681/

[4942682] multimds/basic/{0-supported-random-distro$/{rhel_8.yaml} begin.yaml clusters/9-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/kclient/{mount.yaml overrides/{distro/testing/{flavor/ubuntu_latest.yaml k-testing.yaml} ms-die-on-skipped.yaml}} objectstore-ec/bluestore-ec-root.yaml overrides/{basic/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} fuse-default-perm-no.yaml} q_check_counter/check_counter.yaml tasks/cfuse_workunit_misc.yaml}
time:   00:51:28
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942682/

[4942684] multimds/thrash/{0-supported-random-distro$/{rhel_8.yaml} begin.yaml ceph-thrash/mds.yaml clusters/9-mds-3-standby.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml msgr-failures/none.yaml objectstore-ec/bluestore-comp.yaml overrides/{fuse-default-perm-no.yaml thrash/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} thrash_debug.yaml} tasks/cfuse_workunit_suites_pjd.yaml}
time:   00:21:34
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942684/

[4942685] multimds/basic/{0-supported-random-distro$/{rhel_8.yaml} begin.yaml clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/yes.yaml mount/fuse.yaml objectstore-ec/filestore-xfs.yaml overrides/{basic/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} fuse-default-perm-no.yaml} q_check_counter/check_counter.yaml tasks/cfuse_workunit_norstats.yaml}
time:   00:25:05
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942685/

[4942668] multimds/verify/{begin.yaml centos_latest.yaml clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/kclient/{k-testing.yaml mount.yaml ms-die-on-skipped.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{fuse-default-perm-no.yaml verify/{frag_enable.yaml mon-debug.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml}} tasks/cfuse_workunit_suites_fsstress.yaml validater/lockdep.yaml}
time:   00:17:00
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942668/

[4942669] multimds/basic/{0-supported-random-distro$/{ubuntu_latest.yaml} begin.yaml clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} inline/no.yaml mount/fuse.yaml objectstore-ec/bluestore-comp-ec-root.yaml overrides/{basic/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} fuse-default-perm-no.yaml} q_check_counter/check_counter.yaml tasks/cfuse_workunit_suites_ffsb.yaml}
time:   01:37:18
info:   http://pulpito.ceph.com/teuthology-2020-04-10_13:15:04-multimds-octopus-testing-basic-smithi/4942669/
2020-04-12T17:47:09.513 INFO:teuthology.results:starting coverage generation
2020-04-12T17:47:09.528 ERROR:teuthology.results:error generating memo/results
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 35, in main
    int(args['--timeout']), args['--dry-run'])
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 84, in results
    generate_coverage(archive_dir, name)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 107, in generate_coverage
    archive_dir,
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory