2020-05-09T10:53:32.010 INFO:root:teuthology version: 1.0.0-28c1ab59
2020-05-09T10:53:32.013 INFO:teuthology.results:subset: '1234/9999'
2020-05-09T10:53:32.013 INFO:teuthology.results:seed: '2477'
2020-05-09T11:06:31.279 INFO:root:teuthology version: 1.0.0-28c1ab59
2020-05-09T11:06:31.305 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-05-09T12:22:32.829 INFO:teuthology.results:Tests finished! gathering results...
2020-05-09T12:22:32.900 INFO:teuthology.results:Sending results to tchaikov@gmail.com: Test Run: kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/
failed:  29
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[5038756]  fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/mimic.yaml}}
-----------------------------------------------------------------
time:   00:25:38
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038756/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038756/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=70447adbbab64ed09ebbce879f0b8f2e

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038757]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/no.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:32
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038757/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038757/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=682bbf83dcd343b297b5e67d44653a72

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038758]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/yes.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:42
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038758/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038758/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=7163e7060c714d7d8cb2814fd5bbee71

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038759]  fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-client-upgrade.yaml 4-compat_client.yaml 5-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:30
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038759/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038759/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=32fb970ac63c4fe1a503217c326cbfe1

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038760]  fs/basic_functional/{begin.yaml clusters/1-mds-4-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore/bluestore-ec-root.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{centos_latest.yaml} tasks/client-limits.yaml}
-----------------------------------------------------------------
time:   00:19:46
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038760/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038760/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=a786dd040c024be494dbc6fb26c7853b

    Test failure: test_client_pin
    (tasks.cephfs.test_client_limits.TestClientLimits)

[5038761]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/no.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:37
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038761/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038761/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=ad71f9d412104ae5bc4719cf2ba6f0cf

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038762]  fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/no.yaml}}
-----------------------------------------------------------------
time:   00:25:35
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038762/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038762/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=1926ed145eca47c6b9a30f41de120dbb

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038763]  fs/thrash/{begin.yaml ceph-thrash/default.yaml clusters/1-mds-1-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml msgr-failures/osd-mds-delay.yaml objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_snaptests.yaml}
-----------------------------------------------------------------
time:   00:09:02
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038763/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038763/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=a58681ccfb1342cc8edfa9bc2d3cd6f8

    Command failed on smithi027 with status 128: 'rm -rf
    /home/ubuntu/cephtest/clone.client.0 && git clone
    https://github.com/tchaikov/ceph.git /home/ubuntu/cephtest/clone.client.0
    && cd /home/ubuntu/cephtest/clone.client.0 && git checkout
    435f86602f42800559492d3c67bfdac2c92df0fa'

[5038764]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/yes.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:26:09
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038764/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038764/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=c6a650adaa0d4fbc9641b6d951e90515

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038765]  fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/mimic.yaml}}
-----------------------------------------------------------------
time:   00:25:38
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038765/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038765/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=70d8c97b4f5040c0a933a04ea75d3826

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038766]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/no.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:06
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038766/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038766/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=cb4405b0be5d45518342ecbd71717f38

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038767]  fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-client-upgrade.yaml 4-compat_client.yaml 5-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:25
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038767/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038767/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=72e5e2f63a9e4937bbff8a5c0166bb78

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038768]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/yes.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:24
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038768/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038768/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=27eb81b930e14622ad68edb4e8eadfd9

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038769]  fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/no.yaml}}
-----------------------------------------------------------------
time:   00:25:24
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038769/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038769/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=768ac0b8de8f4b41ae5e773a9262a512

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038770]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/no.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:38
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038770/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038770/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=9f193a80fed34b2890faa0b4f4807b7f

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038771]  fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/mimic.yaml}}
-----------------------------------------------------------------
time:   00:25:34
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038771/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038771/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=dd173122b4cb480f90a18ff71932ab5a

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038772]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/yes.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:10:09
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038772/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038772/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=8a26727c253e470a804e3ad27ac0f6bf

    [Errno None] Unable to connect to port 22 on 172.21.15.100

[5038773]  fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-client-upgrade.yaml 4-compat_client.yaml 5-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:24:51
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038773/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038773/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=ad6deea111c8447a9a8c1e735a5b0764

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038774]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/no.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:55
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038774/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038774/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=bfb60d92befe4dd88929b9be962b7c60

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038775]  fs/basic_functional/{begin.yaml clusters/1-mds-4-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml objectstore/bluestore-bitmap.yaml overrides/{frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_16.04.yaml} tasks/client-limits.yaml}
-----------------------------------------------------------------
time:   00:15:27
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038775/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038775/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=edb75d09dfbe468a8c7e6b16b28e7b3c

    Test failure: test_client_pin
    (tasks.cephfs.test_client_limits.TestClientLimits)

[5038776]  fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/no.yaml}}
-----------------------------------------------------------------
time:   00:25:33
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038776/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038776/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=29881667c9884ea880c7bcb8c52c0f8a

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038777]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/filestore-xfs.yaml overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/yes.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:28
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038777/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038777/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=7e57ab8c9d60483a88156bd452e656f0

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038778]  fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/mimic.yaml}}
-----------------------------------------------------------------
time:   00:03:34
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038778/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038778/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=5aab0db8e24b4fdea73f663ef685e624

    psutil.NoSuchProcess process no longer exists (pid=2315)

[5038779]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-bitmap.yaml overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/no.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:38
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038779/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038779/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=cf4f78a91a6b49f0aedc2ccf8bffac36

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038780]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp-ec-root.yaml overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/yes.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:33
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038780/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038780/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=991702b956024ce791f335408dfb564b

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038781]  fs/thrash/{begin.yaml ceph-thrash/default.yaml clusters/1-mds-1-client-coloc.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} mount/fuse.yaml msgr-failures/osd-mds-delay.yaml objectstore-ec/bluestore-ec-root.yaml overrides/{frag_enable.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} supported-random-distros$/{ubuntu_latest.yaml} tasks/cfuse_workunit_snaptests.yaml}
-----------------------------------------------------------------
time:   00:09:01
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038781/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038781/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=91986130475745a3bedabbdfeddb2dcb

    Command failed on smithi043 with status 128: 'rm -rf
    /home/ubuntu/cephtest/clone.client.0 && git clone
    https://github.com/tchaikov/ceph.git /home/ubuntu/cephtest/clone.client.0
    && cd /home/ubuntu/cephtest/clone.client.0 && git checkout
    435f86602f42800559492d3c67bfdac2c92df0fa'

[5038782]  fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-client-upgrade.yaml 4-compat_client.yaml 5-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:33
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038782/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038782/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=48fd896c5d0f4f3492e6c8e598b0aae7

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038783]  fs/upgrade/snaps/{clusters/3-mds.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} objectstore-ec/bluestore-comp.yaml overrides/{frag_enable.yaml multimds/yes.yaml whitelist_health.yaml whitelist_rstat.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-sanity.yaml 4-client-upgrade/no.yaml 5-client-sanity.yaml 6-snap-upgrade.yaml 7-client-sanity.yaml}}
-----------------------------------------------------------------
time:   00:25:28
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038783/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038783/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=adc91b489cf744d68e9a1419b95633b5

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds

[5038784]  fs/upgrade/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/no.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-luminous.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/no.yaml}}
-----------------------------------------------------------------
time:   00:25:35
info:   http://pulpito.ceph.com/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038784/
log:    http://qa-proxy.ceph.com/teuthology/kchai-2020-05-09_07:01:38-fs-nautilus-distro-basic-smithi/5038784/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=4e5b30b681c0487983fb498d7486dff1

    'wait_until_healthy' reached maximum tries (150) after waiting for 900
    seconds
2020-05-09T12:22:32.985 INFO:teuthology.results:starting coverage generation
2020-05-09T12:22:32.993 ERROR:teuthology.results:error generating memo/results
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 35, in main
    int(args['--timeout']), args['--dry-run'])
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 84, in results
    generate_coverage(archive_dir, name)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 107, in generate_coverage
    archive_dir,
  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage': '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage'