2020-09-17T14:24:16.569 INFO:root:teuthology version: 1.0.0-55bd2dc 2020-09-17T14:24:16.571 INFO:teuthology.results:seed: '2113' 2020-09-17T14:40:22.736 INFO:root:teuthology version: 1.0.0-55bd2dc 2020-09-17T14:40:22.771 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-09-18T02:17:35.014 INFO:teuthology.results:Tests finished! gathering results... 2020-09-18T02:17:42.328 INFO:scrape:Found 16 jobs 2020-09-18T02:17:44.060 INFO:scrape:Found 1 distinct failure reasons 2020-09-18T02:17:44.061 INFO:scrape:Failure: "2020-09-17T15:40:23.027113+0000 mds.a (mds.0) 5 : cluster [WRN] evicting unresponsive client smithi115:x (54268), after waiting 48.4471 seconds during MDS startup" in cluster log 2020-09-18T02:17:44.061 INFO:scrape:1 jobs: ['5443745'] 2020-09-18T02:17:44.061 INFO:scrape:suites: ['1-ceph-install/nautilus', '1.1-pg-log-overrides/short_pg_log', '2-workload/{blogbench', '3-upgrade-sequence/upgrade-mon-osd-mds', '4-octopus', '5-final-workload/{blogbench', 'ec-rados-default', 'objectstore/filestore-xfs', 'rados-snaps-few-objects', 'rados_api', 'rados_loadgenbig', 'rados_loadgenmix', 'rados_mon_thrash', 'rbd_cls', 'rbd_import_export', 'rgw', 'rgw_ragweed_check}', 'rgw_ragweed_prepare', 'start}', 'test_rbd_api', 'test_rbd_python}', 'ubuntu_18.04}', 'upgrade:nautilus-x/parallel/{0-cluster/{openstack'] 2020-09-18T02:17:44.061 INFO:scrape: 2020-09-18T02:17:44.064 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/ failed: 1 dead: 0 running: 0 waiting: 0 queued: 0 passed: 15 Fail ================================================================= [5443745] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} ----------------------------------------------------------------- time: 02:17:57 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443745/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443745/ "2020-09-17T15:40:23.027113+0000 mds.a (mds.0) 5 : cluster [WRN] evicting unresponsive client smithi115:x (54268), after waiting 48.4471 seconds during MDS startup" in cluster log Pass ================================================================= [5443742] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_18.04} time: 02:32:15 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443742/ [5443743] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04} time: 04:33:48 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443743/ [5443744] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_18.04} time: 03:26:52 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443744/ [5443746] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_18.04} time: 02:28:54 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443746/ [5443747] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_18.04} time: 03:18:43 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443747/ [5443748] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_18.04} time: 01:37:45 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443748/ [5443749] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} time: 02:23:27 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443749/ [5443750] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_18.04} time: 02:29:07 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443750/ [5443751] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_18.04} time: 04:46:52 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443751/ [5443752] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-bitmap 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_18.04} time: 03:26:21 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443752/ [5443753] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} time: 02:17:55 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443753/ [5443754] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_18.04} time: 02:33:40 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443754/ [5443755] upgrade:nautilus-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_18.04} time: 03:35:42 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443755/ [5443756] upgrade:nautilus-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-nautilus-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-octopus 7-final-workload thrashosds-health ubuntu_18.04} time: 01:58:54 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443756/ [5443757] upgrade:nautilus-x/parallel/{0-cluster/{openstack start} 1-ceph-install/nautilus 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_18.04} time: 02:15:24 info: http://pulpito.ceph.com/teuthology-2020-09-17_14:23:03-upgrade:nautilus-x-master-distro-basic-smithi/5443757/