2020-05-02T07:43:44.298 INFO:root:teuthology version: 1.0.0-58684fc5
2020-05-02T07:43:44.300 INFO:teuthology.results:seed: '2517'
2020-05-02T09:16:13.398 INFO:root:teuthology version: 1.0.0-58684fc5
2020-05-02T09:16:13.403 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-05-02T21:16:24.696 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds
2020-05-02T21:16:24.768 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/
failed:  18
dead:    4
running: 0
waiting: 0
queued:  0
passed:  2


Fail
=================================================================
[5014862]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{centos_8.yaml} tasks/cfuse_workunit_suites_blogbench.yaml}
-----------------------------------------------------------------
time:   00:10:18
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014862/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014862/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=d443bd8110d840a7aaf005e9626c370a

    Command failed on smithi038 with status 4: "sudo bash -c 'iptables -A
    FORWARD -o ens1f0 -i ceph-brx -j ACCEPT'"

[5014863]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{rhel_8.yaml} tasks/cfuse_workunit_suites_fsstress.yaml}
-----------------------------------------------------------------
time:   00:18:29
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014863/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014863/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=6b2c7a183691404aa205140f8592cd2b

    "2020-05-02T09:01:44.482213+0000 mon.a (mon.1) 2 : cluster [WRN] message
    from mon.0 was stamped 1.198297s in the future, clocks not synchronized" in
    cluster log

[5014864]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{rhel_8.yaml} tasks/cfuse_workunit_suites_iozone.yaml}
-----------------------------------------------------------------
time:   00:12:10
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014864/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014864/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=4db8f8fb78054db6a7d08838358639a3

    Command failed on smithi101 with status 4: "sudo bash -c 'iptables -A
    FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT'"

[5014865]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{rhel_8.yaml} tasks/cfuse_workunit_suites_pjd.yaml}
-----------------------------------------------------------------
time:   00:12:14
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014865/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014865/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=e9ef7f3d5ffb49f0ab2ba222003ed5e9

    Command failed on smithi150 with status 4: "sudo bash -c 'iptables -A
    FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT'"

[5014866]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/kclient_workunit_direct_io.yaml}
-----------------------------------------------------------------
time:   11:50:42
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014866/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014866/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=40d9ec4f6e1f419bb2f3bbfce0f1c08f

    psutil.NoSuchProcess process no longer exists (pid=673)

[5014867]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{centos_8.yaml} tasks/kclient_workunit_suites_dbench.yaml}
-----------------------------------------------------------------
time:   00:10:16
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014867/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014867/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=77a2a29da1714627b4a41143f5940d54

    Command failed on smithi013 with status 4: "sudo bash -c 'iptables -A
    FORWARD -o ens1f0 -i ceph-brx -j ACCEPT'"

[5014868]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/kclient_workunit_suites_fsstress.yaml}
-----------------------------------------------------------------
time:   11:24:16
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014868/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014868/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=716c0ec1bbda47eb8bd2c0d45f8f2c40

    psutil.NoSuchProcess process no longer exists (pid=31724)

[5014869]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{rhel_8.yaml} tasks/kclient_workunit_suites_pjd.yaml}
-----------------------------------------------------------------
time:   00:12:05
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014869/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014869/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=6578da9777694034b58d6e81b6bb9b59

    Command failed on smithi017 with status 4: "sudo bash -c 'iptables -A
    FORWARD -o ens1f0 -i ceph-brx -j ACCEPT'"

[5014870]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{centos_8.yaml} tasks/libcephfs_interface_tests.yaml}
-----------------------------------------------------------------
time:   00:10:18
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014870/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014870/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=7a06ed8270af4a30b53e4eff3ca3a060

    Command failed on smithi042 with status 4: "sudo bash -c 'iptables -A
    FORWARD -o ens1f0 -i ceph-brx -j ACCEPT'"

[5014872]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/rados_api_tests.yaml}
-----------------------------------------------------------------
time:   11:45:08
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014872/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014872/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=eedd28ed5fcc4325b8807896a21a7a71

    psutil.NoSuchProcess process no longer exists (pid=20982)

[5014873]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/rados_bench.yaml}
-----------------------------------------------------------------
time:   11:41:57
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014873/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014873/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=15d1c57abc4046f5ab7cd621c925f0b5

    psutil.NoSuchProcess process no longer exists (pid=31874)

[5014877]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/rados_python.yaml}
-----------------------------------------------------------------
time:   11:31:53
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014877/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014877/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=e059c6241bf64cba91e62a2987276bc0

    psutil.NoSuchProcess process no longer exists (pid=1827)

[5014878]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/rados_workunit_loadgen_mix.yaml}
-----------------------------------------------------------------
time:   11:42:33
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014878/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014878/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=4ce0c09b4e204b8cb8f944fef36542a4

    psutil.NoSuchProcess process no longer exists (pid=5158)

[5014879]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/rbd_api_tests.yaml}
-----------------------------------------------------------------
time:   11:40:06
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014879/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014879/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=217ddf7668e2466590e912fd36db39f1

    psutil.NoSuchProcess process no longer exists (pid=17027)

[5014880]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{rhel_8.yaml} tasks/rbd_cli_import_export.yaml}
-----------------------------------------------------------------
time:   00:12:04
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014880/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014880/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=6c31c20b94fd4f6b86def12acd0a7919

    Command failed on smithi140 with status 4: "sudo bash -c 'iptables -A
    FORWARD -o ens1f1 -i ceph-brx -j ACCEPT'"

[5014881]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{centos_8.yaml} tasks/rbd_fsx.yaml}
-----------------------------------------------------------------
time:   00:15:13
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014881/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014881/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=38af03cec54c4bf09e968365ba7ca499

    "2020-05-02T10:37:35.531645+0000 mon.a (mon.1) 2 : cluster [WRN] message
    from mon.0 was stamped 3.346116s in the future, clocks not synchronized" in
    cluster log

[5014882]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{centos_8.yaml} tasks/rbd_python_api_tests.yaml}
-----------------------------------------------------------------
time:   00:09:59
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014882/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014882/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=69b2cc0634cb4e0b8c014549769767bf

    Command failed on smithi072 with status 4: "sudo bash -c 'iptables -A
    FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT'"

[5014883]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{rhel_8.yaml} tasks/rbd_workunit_suites_iozone.yaml}
-----------------------------------------------------------------
time:   00:17:16
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014883/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014883/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=ec7afd7d8e094230888acda93bdbc528

    "2020-05-02T11:48:05.864958+0000 mon.b (mon.1) 2 : cluster [WRN] message
    from mon.0 was stamped 2.394803s in the future, clocks not synchronized" in
    cluster log



Dead
=================================================================
[5014874]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{rhel_8.yaml} tasks/rados_cache_snaps.yaml}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014874/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014874/

[5014875]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/rados_cls_all.yaml}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014875/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014875/

[5014876]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{ubuntu_latest.yaml} tasks/rados_ec_snaps.yaml}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014876/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014876/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=aa0685c57ca04248822464f7c975180b

    reached maximum tries (60) after waiting for 900 seconds

[5014884]  smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{rhel_8.yaml} tasks/rgw_ec_s3tests.yaml}
-----------------------------------------------------------------
time:   00:00:00
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014884/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014884/



Pass
=================================================================
[5014871] smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{centos_8.yaml} tasks/mon_thrash.yaml}
time:   00:27:48
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014871/

[5014885] smoke/basic/{clusters/{fixed-3-cephfs.yaml openstack.yaml} objectstore/bluestore-bitmap.yaml supported-random-distro$/{rhel_8.yaml} tasks/rgw_s3tests.yaml}
time:   00:22:39
info:   http://pulpito.ceph.com/teuthology-2020-05-02_05:00:02-smoke-master-testing-basic-smithi/5014885/
2020-05-02T21:16:24.826 INFO:teuthology.results:starting coverage generation
2020-05-02T21:16:24.837 ERROR:teuthology.results:error generating memo/results
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 35, in main
    int(args['--timeout']), args['--dry-run'])
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 84, in results
    generate_coverage(archive_dir, name)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 107, in generate_coverage
    archive_dir,
  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage': '/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-coverage'