2020-02-15T18:15:40.416 INFO:root:teuthology version: 1.0.0-7f4dc17
2020-02-15T18:15:40.417 INFO:teuthology.results:subset: '100/9000'
2020-02-15T18:15:40.417 INFO:teuthology.results:seed: '3022'
2020-02-15T18:25:24.245 INFO:root:teuthology version: 1.0.0-7f4dc17
2020-02-15T18:25:24.282 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish...
2020-02-16T06:25:37.015 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds
2020-02-16T06:25:37.067 INFO:teuthology.results:Sending results to dzafman@redhat.com: Test Run: dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi
=================================================================
info:    http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/
logs:    http://qa-proxy.ceph.com/teuthology/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/
failed:  3
dead:    0
running: 1
waiting: 0
queued:  0
passed:  12


Fail
=================================================================
[4768926]  upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/rhel_7.yaml thrashosds-health.yaml}
-----------------------------------------------------------------
time:   05:46:42
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768926/
log:    http://qa-proxy.ceph.com/teuthology/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768926/

    Command failed on smithi017 with status 124: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster
    ceph tell osd.2 flush_pg_stats'

[4768932]  upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml}
-----------------------------------------------------------------
time:   05:45:51
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768932/
log:    http://qa-proxy.ceph.com/teuthology/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768932/

    Command failed on smithi152 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster
    ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok
    dump_recovery_reservations'

[4768929]  upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_16.04.yaml thrashosds-health.yaml}
-----------------------------------------------------------------
time:   03:30:25
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768929/
log:    http://qa-proxy.ceph.com/teuthology/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768929/

    Command failed on smithi163 with status 22: 'sudo adjust-ulimits ceph-
    coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster
    ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok
    dump_recovery_reservations'



Running
=================================================================
[4768938] upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/rhel_7.yaml thrashosds-health.yaml}
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768938/



Pass
=================================================================
[4768941] upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_16.04.yaml thrashosds-health.yaml}
time:   05:26:47
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768941/

[4768940] upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_16.04.yaml}
time:   02:07:57
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768940/

[4768934] upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml}
time:   02:15:43
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768934/

[4768939] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/rhel_7.yaml thrashosds-health.yaml}
time:   00:51:58
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768939/

[4768928] upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_16.04.yaml}
time:   02:19:30
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768928/

[4768927] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/rhel_7.yaml thrashosds-health.yaml}
time:   00:58:32
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768927/

[4768935] upgrade:mimic-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift-ragweed_check.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml}
time:   05:23:28
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768935/

[4768936] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml}
time:   00:50:27
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768936/

[4768937] upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/rhel_7.yaml}
time:   02:28:03
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768937/

[4768930] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_16.04.yaml thrashosds-health.yaml}
time:   00:44:06
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768930/

[4768931] upgrade:mimic-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/mimic.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml}
time:   02:11:21
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768931/

[4768933] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/mimic.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml}
time:   00:44:08
info:   http://pulpito.ceph.com/dzafman-2020-02-15_10:14:53-upgrade:mimic-x-master-distro-basic-smithi/4768933/
2020-02-16T06:25:37.151 INFO:teuthology.results:starting coverage generation