2020-06-27T02:30:51.767 INFO:root:teuthology version: 1.0.0-ebb68fe
2020-06-27T02:30:51.769 INFO:teuthology.results:seed: '6961'
2020-06-27T02:37:16.209 INFO:root:teuthology version: 1.0.0-ebb68fe
2020-06-27T02:37:16.238 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-06-27T03:27:17.309 INFO:teuthology.results:Tests finished! gathering results...
2020-06-27T03:27:17.375 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/
failed:  16
dead:    0
running: 0
waiting: 0
queued:  0
passed:  0


Fail
=================================================================
[5182637]  upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest}
-----------------------------------------------------------------
time:   00:05:54
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182637/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182637/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=150167be00ed46a4bea964c5f4947597

    Command failed on smithi067 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182638]  upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_latest}
-----------------------------------------------------------------
time:   00:06:39
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182638/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182638/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=9520bbda43654710aee87f5fb96dc7a9

    Command failed on smithi141 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182639]  upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/bluestore-bitmap thrashosds-health ubuntu_latest}
-----------------------------------------------------------------
time:   00:06:31
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182639/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182639/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=f2569c8e53a24d2aa7f743b6c5027e39

    Command failed on smithi073 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182640]  upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest}
-----------------------------------------------------------------
time:   00:05:39
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182640/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182640/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=e323ec2b2b4d43c4b3163cfcc5cc94fb

    Command failed on smithi196 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182641]  upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest}
-----------------------------------------------------------------
time:   00:05:51
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182641/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182641/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=0234320f97394d59af3f90237089fe55

    Command failed on smithi169 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182642]  upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_latest}
-----------------------------------------------------------------
time:   00:06:03
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182642/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182642/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=b851b6ca011e41be91ffba4c21204bbb

    Command failed on smithi194 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182643]  upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/filestore-xfs thrashosds-health ubuntu_latest}
-----------------------------------------------------------------
time:   00:06:19
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182643/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182643/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=9e7e396e41e8492a905bd5369b9018c2

    Command failed on smithi062 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182644]  upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest}
-----------------------------------------------------------------
time:   00:05:43
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182644/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182644/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=de4f3aab13b74eb488b510d52399686f

    Command failed on smithi107 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182645]  upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest}
-----------------------------------------------------------------
time:   00:05:51
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182645/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182645/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=7d537256a3e148179c36c06d49f86cf4

    Command failed on smithi058 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182646]  upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_latest}
-----------------------------------------------------------------
time:   00:06:43
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182646/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182646/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=fe7a7c1e51764bcd8a84f73ff9a07635

    Command failed on smithi172 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182647]  upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/bluestore-bitmap thrashosds-health ubuntu_latest}
-----------------------------------------------------------------
time:   00:06:32
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182647/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182647/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=940262075002478ebbdbf239237bb7c4

    Command failed on smithi171 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182648]  upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest}
-----------------------------------------------------------------
time:   00:05:39
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182648/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182648/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=c1eaf18a03954cc888f62ba46394bda3

    Command failed on smithi035 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182649]  upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest}
-----------------------------------------------------------------
time:   00:05:57
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182649/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182649/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=9f69abb4102049179addcb01f378f1a4

    Command failed on smithi189 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182650]  upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_latest}
-----------------------------------------------------------------
time:   00:06:08
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182650/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182650/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=5d454dbadea043b5bb6ede0800b4909c

    Command failed on smithi052 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182651]  upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/filestore-xfs thrashosds-health ubuntu_latest}
-----------------------------------------------------------------
time:   00:06:15
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182651/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182651/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=55312c5ab09e4810a3eadb27c26f1099

    Command failed on smithi175 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'

[5182652]  upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest}
-----------------------------------------------------------------
time:   00:05:40
info:   http://pulpito.ceph.com/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182652/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2020-06-27_02:30:03-upgrade:mimic-x-octopus-distro-basic-smithi/5182652/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=6377a43317da44a9936641ee70c264cc

    Command failed on smithi114 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph --log-early osd dump --format=json'