2020-06-19T12:11:58.018 INFO:root:teuthology version: 1.0.0-4ff7700 2020-06-19T12:11:58.020 INFO:teuthology.results:subset: '822/9999' 2020-06-19T12:11:58.021 INFO:teuthology.results:seed: '2477' 2020-06-19T12:17:37.267 INFO:root:teuthology version: 1.0.0-4ff7700 2020-06-19T12:17:37.273 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-06-19T17:42:43.764 INFO:teuthology.results:Tests finished! gathering results... 2020-06-19T17:42:43.825 INFO:teuthology.results:Sending results to tchaikov@gmail.com: Test Run: kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 19 Fail ================================================================= [5163969] upgrade/nautilus-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-octopus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml} objectstore/filestore-xfs.yaml ubuntu_18.04.yaml} ----------------------------------------------------------------- time: 02:13:42 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163969/ log: http://qa-proxy.ceph.com/teuthology/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163969/ "2020-06-19T12:42:34.963814+0000 mds.a (mds.0) 5 : cluster [WRN] evicting unresponsive client smithi029:x (34303), after waiting 48.4992 seconds during MDS startup" in cluster log Pass ================================================================= [5163963] upgrade/nautilus-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-octopus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml} objectstore/bluestore-bitmap.yaml ubuntu_18.04.yaml} time: 02:19:28 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163963/ [5163964] upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/no.yaml pg-warn.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-nautilus.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/no.yaml}} time: 00:31:38 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163964/ [5163965] upgrade/nautilus-x-singleton/{0-cluster/{openstack.yaml start.yaml} 1-install/nautilus.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{rbd-cls.yaml rbd-import-export.yaml readwrite.yaml snaps-few-objects.yaml} 5-workload/{radosbench.yaml rbd_api.yaml} 6-finish-upgrade.yaml 7-octopus.yaml 8-workload/{rbd-python.yaml snaps-many-objects.yaml} bluestore-bitmap.yaml thrashosds-health.yaml ubuntu_latest.yaml} time: 03:45:05 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163965/ [5163966] upgrade/nautilus-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-msgr2.yaml 8-final-workload/{rbd-python.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml thrashosds-health.yaml ubuntu_18.04.yaml} time: 03:33:42 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163966/ [5163967] upgrade/nautilus-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-nautilus-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 3.1-objectstore/bluestore-bitmap.yaml 4-ec-workload/{rados-ec-workload.yaml rbd-ec-workload.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-final-workload.yaml thrashosds-health.yaml ubuntu_18.04.yaml} time: 02:17:13 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163967/ [5163968] upgrade/cephfs/volumes/import-legacy/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml pg-warn.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-nautilus.yaml 1-client.yaml 2-upgrade.yaml 3-verify.yaml} ubuntu_18.04.yaml} time: 01:14:52 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163968/ [5163970] upgrade/cephfs/featureful_client/upgraded_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/no.yaml pg-warn.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-nautilus.yaml 1-client.yaml 2-upgrade.yaml 3-client-upgrade.yaml 4-compat_client.yaml 5-client-sanity.yaml}} time: 00:19:12 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163970/ [5163971] upgrade/nautilus-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-msgr2.yaml 8-final-workload/{rbd-python.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml thrashosds-health.yaml ubuntu_18.04.yaml} time: 03:38:58 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163971/ [5163972] upgrade/nautilus-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-nautilus-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 3.1-objectstore/filestore-xfs.yaml 4-ec-workload/{rados-ec-workload.yaml rbd-ec-workload.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-final-workload.yaml thrashosds-health.yaml ubuntu_18.04.yaml} time: 01:41:31 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163972/ [5163973] upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/yes.yaml pg-warn.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-nautilus.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/pacific.yaml}} time: 00:22:23 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163973/ [5163974] upgrade/nautilus-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-octopus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml} objectstore/bluestore-bitmap.yaml ubuntu_18.04.yaml} time: 02:20:01 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163974/ [5163975] upgrade/nautilus-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-msgr2.yaml 8-final-workload/{rbd-python.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml thrashosds-health.yaml ubuntu_18.04.yaml} time: 04:02:03 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163975/ [5163976] upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/yes.yaml pg-warn.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-nautilus.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/no.yaml}} time: 00:31:00 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163976/ [5163977] upgrade/nautilus-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-nautilus-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 3.1-objectstore/bluestore-bitmap.yaml 4-ec-workload/{rados-ec-workload.yaml rbd-ec-workload.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-final-workload.yaml thrashosds-health.yaml ubuntu_18.04.yaml} time: 01:59:58 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163977/ [5163978] upgrade/cephfs/featureful_client/upgraded_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/yes.yaml pg-warn.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-nautilus.yaml 1-client.yaml 2-upgrade.yaml 3-client-upgrade.yaml 4-compat_client.yaml 5-client-sanity.yaml}} time: 00:20:24 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163978/ [5163979] upgrade/nautilus-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-octopus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml} objectstore/filestore-xfs.yaml ubuntu_18.04.yaml} time: 02:12:24 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163979/ [5163980] upgrade/nautilus-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml rgw_ragweed_prepare.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-msgr2.yaml 8-final-workload/{rbd-python.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml thrashosds-health.yaml ubuntu_18.04.yaml} time: 03:16:38 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163980/ [5163981] upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap.yaml clusters/1-mds-2-client-micro.yaml conf/{client.yaml mds.yaml mon.yaml osd.yaml} overrides/{frag_enable.yaml multimds/no.yaml pg-warn.yaml whitelist_health.yaml whitelist_wrongly_marked_down.yaml} tasks/{0-nautilus.yaml 1-client.yaml 2-upgrade.yaml 3-compat_client/pacific.yaml}} time: 00:22:27 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163981/ [5163982] upgrade/nautilus-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-nautilus-install/nautilus.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 3.1-objectstore/filestore-xfs.yaml 4-ec-workload/{rados-ec-workload.yaml rbd-ec-workload.yaml} 5-finish-upgrade.yaml 6-octopus.yaml 7-final-workload.yaml thrashosds-health.yaml ubuntu_18.04.yaml} time: 01:53:47 info: http://pulpito.ceph.com/kchai-2020-06-19_07:01:33-upgrade-wip-kefu-testing-2020-06-19-1102-distro-basic-smithi/5163982/