2020-04-20T02:26:07.244 INFO:root:teuthology version: 1.0.0-9ce0dec 2020-04-20T02:26:07.244 INFO:teuthology.results:seed: '4230' 2020-04-20T03:00:49.439 INFO:root:teuthology version: 1.0.0-9ce0dec 2020-04-20T03:00:49.451 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-04-20T07:55:54.498 INFO:teuthology.results:Tests finished! gathering results... 2020-04-20T07:55:54.588 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/ failed: 32 dead: 0 running: 0 waiting: 0 queued: 0 passed: 0 Fail ================================================================= [4967768] upgrade:luminous-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:25:48 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967768/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967768/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=423a596ffa5b4953bd2a7c1ed85f70e9 'pg_num_target' [4967769] upgrade:luminous-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:25:41 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967769/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967769/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=1975cd24276546768e68ebe16a7324ff 'pg_num_target' [4967763] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml} ----------------------------------------------------------------- time: 00:29:20 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967763/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967763/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=db19ce92dfa94cada519968cf244b607 "2020-04-20 02:50:38.322600 mon.c mon.0 172.21.15.33:6789/0 138 : cluster [WRN] Health check failed: too few PGs per OSD (2 < min 30) (TOO_FEW_PGS)" in cluster log [4967766] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml} ----------------------------------------------------------------- time: 00:25:19 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967766/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967766/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=67f84fca019b42338c49ddadf77d240d "2020-04-20 02:46:48.572267 mon.c mon.0 172.21.15.77:6789/0 139 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967767] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml} ----------------------------------------------------------------- time: 00:29:05 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967767/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967767/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=9536b17478c145b1b166579ae9356414 "2020-04-20 03:56:01.489583 mon.b mon.0 172.21.15.8:6789/0 128 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967764] upgrade:luminous-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:33:00 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967764/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967764/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=d28ff2e505b540cda715682913447e9b 'pg_num_target' [4967765] upgrade:luminous-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:29:19 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967765/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967765/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=13f46440a00c43ea90a32a89f962d502 'pg_num_target' [4967784] upgrade:luminous-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:25:31 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967784/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967784/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=9f26834ce2194b3ebedae65e25eb7a52 'pg_num_target' [4967785] upgrade:luminous-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:25:35 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967785/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967785/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=790d321d270b49c6806127bdabb3cef3 'pg_num_target' [4967786] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml} ----------------------------------------------------------------- time: 00:25:57 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967786/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967786/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=4bab3b888f87457196ccce73596cdc5f "2020-04-20 03:08:16.161337 mon.c mon.0 172.21.15.78:6789/0 138 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967787] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml} ----------------------------------------------------------------- time: 00:29:03 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967787/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967787/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=31ac7fdc43874135a09f9714e06ff8e7 "2020-04-20 03:30:32.594304 mon.a mon.0 172.21.15.82:6789/0 267 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967780] upgrade:luminous-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:29:30 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967780/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967780/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=3a8bf522e2084986affd7eb5e66156b6 'pg_num_target' [4967781] upgrade:luminous-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:33:12 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967781/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967781/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=82a42e6b96694b54970cc2224401c946 'pg_num_target' [4967782] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml} ----------------------------------------------------------------- time: 00:25:26 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967782/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967782/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=2420d634991747859e6078a1b41323f3 "2020-04-20 03:02:50.084797 mon.c mon.0 172.21.15.98:6789/0 140 : cluster [WRN] Health check failed: too few PGs per OSD (2 < min 30) (TOO_FEW_PGS)" in cluster log [4967783] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml} ----------------------------------------------------------------- time: 00:28:47 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967783/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967783/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=8f0763a0fb1d45eb981fa96d69736552 "2020-04-20 03:58:12.137966 mon.c mon.0 172.21.15.44:6789/0 260 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967788] upgrade:luminous-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:29:18 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967788/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967788/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=857975bda97c4e71ae26a38470b8cde9 'pg_num_target' [4967789] upgrade:luminous-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:29:07 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967789/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967789/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=5830e48866cf4e989bb366565d47ca4f 'pg_num_target' [4967779] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml} ----------------------------------------------------------------- time: 00:29:28 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967779/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967779/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=72cdf64fb4f64b89bb932f056c54a40e "2020-04-20 03:56:27.606999 mon.a mon.0 172.21.15.68:6789/0 279 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967778] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml} ----------------------------------------------------------------- time: 00:25:49 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967778/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967778/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=7242ecd8bf244b109e2734a3111a1237 "2020-04-20 03:17:31.304843 mon.b mon.0 172.21.15.23:6789/0 276 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967775] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml} ----------------------------------------------------------------- time: 00:28:35 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967775/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967775/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=e2b51778fd354b169ec91de4d461b145 "2020-04-20 03:22:03.413329 mon.a mon.0 172.21.15.103:6789/0 269 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967774] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml} ----------------------------------------------------------------- time: 00:25:28 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967774/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967774/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=48ab3d4e9ee840cf87500710104b5c13 "2020-04-20 03:15:21.266319 mon.a mon.0 172.21.15.51:6789/0 261 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967777] upgrade:luminous-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:26:10 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967777/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967777/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=d3fbc7ca8b09469198a4bbfc6428592c 'pg_num_target' [4967776] upgrade:luminous-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:25:59 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967776/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967776/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=4f176d7266d04460b9db93c973ef0b04 'pg_num_target' [4967771] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/centos_latest.yaml} ----------------------------------------------------------------- time: 00:29:13 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967771/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967771/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=ef477dbd6719455d966ab6ecae03ca80 "2020-04-20 04:05:17.918867 mon.b mon.0 172.21.15.61:6789/0 130 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967770] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml} ----------------------------------------------------------------- time: 00:25:50 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967770/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967770/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=f59c8f94dbc74a72b7357450eace73bf "2020-04-20 02:49:54.739415 mon.c mon.0 172.21.15.1:6789/0 140 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967773] upgrade:luminous-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:29:01 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967773/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967773/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=196285c3965e48cfb3e872617c476ac9 'pg_num_target' [4967772] upgrade:luminous-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:28:57 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967772/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967772/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=8512df821ef643b588c90529a73c4cb9 'pg_num_target' [4967794] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml} ----------------------------------------------------------------- time: 00:26:04 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967794/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967794/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=025a988652334287b8c8ba4a784009e5 "2020-04-20 03:21:41.453244 mon.c mon.0 172.21.15.13:6789/0 274 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967793] upgrade:luminous-x/stress-split-erasure-code/{0-cluster/{openstack.yaml start.yaml} 1-luminous-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-ec-workload.yaml 5-finish-upgrade.yaml 7-final-workload.yaml objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:26:04 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967793/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967793/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=c4470f91441840cca12e5f8827356c96 'pg_num_target' [4967792] upgrade:luminous-x/stress-split/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-partial-upgrade/firsthalf.yaml 3-thrash/default.yaml 4-workload/{radosbench.yaml rbd-cls.yaml rbd-import-export.yaml rbd_api.yaml readwrite.yaml snaps-few-objects.yaml} 5-finish-upgrade.yaml 6-msgr2.yaml 6-nautilus.yaml 7-final-workload/{rbd-python.yaml rgw-swift.yaml snaps-many-objects.yaml} objectstore/bluestore-bitmap.yaml supported-all-distro/ubuntu_latest.yaml thrashosds-health.yaml} ----------------------------------------------------------------- time: 00:25:56 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967792/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967792/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=bbc951842d7d4d08b660e71e0961e899 'pg_num_target' [4967791] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/short_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-mon-osd-mds.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/centos_latest.yaml} ----------------------------------------------------------------- time: 00:28:54 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967791/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967791/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=3f44c4711a2b41b49f28e502b039bd5e "2020-04-20 03:23:16.717739 mon.a mon.0 172.21.15.79:6789/0 136 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log [4967790] upgrade:luminous-x/parallel/{0-cluster/{openstack.yaml start.yaml} 1-ceph-install/luminous.yaml 1.1-pg-log-overrides/normal_pg_log.yaml 2-workload/{blogbench.yaml ec-rados-default.yaml rados_api.yaml rados_loadgenbig.yaml rgw_ragweed_prepare.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-msgr2.yaml 4-nautilus.yaml 5-final-workload/{blogbench.yaml rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw.yaml rgw_ragweed_check.yaml rgw_swift.yaml} objectstore/filestore-xfs.yaml supported-all-distro/ubuntu_latest.yaml} ----------------------------------------------------------------- time: 00:25:16 info: http://pulpito.ceph.com/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967790/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-04-20_02:25:04-upgrade:luminous-x-nautilus-distro-basic-smithi/4967790/ sentry: http://sentry.ceph.com/sepia/teuthology/?q=f2bedbfa2e4c40deb1f903a28e4e1298 "2020-04-20 03:10:00.992876 mon.c mon.0 172.21.15.75:6789/0 132 : cluster [WRN] Health check failed: too few PGs per OSD (1 < min 30) (TOO_FEW_PGS)" in cluster log 2020-04-20T07:55:54.758 INFO:teuthology.results:starting coverage generation 2020-04-20T07:55:54.774 ERROR:teuthology.results:error generating memo/results Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 35, in main int(args['--timeout']), args['--dry-run']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 84, in results generate_coverage(archive_dir, name) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/results.py", line 107, in generate_coverage archive_dir, File "/usr/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory