2020-12-29T02:51:51.173 INFO:root:teuthology version: 1.0.0-8e49dda4 2020-12-29T02:51:51.174 INFO:teuthology.results:seed: '777' 2020-12-29T03:15:50.967 INFO:root:teuthology version: 1.0.0-8e49dda4 2020-12-29T03:15:51.020 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2020-12-29T14:26:02.599 INFO:teuthology.results:Tests finished! gathering results... 2020-12-29T14:26:06.804 INFO:scrape:Found 16 jobs 2020-12-29T14:26:09.750 INFO:scrape:Found 1 distinct failure reasons 2020-12-29T14:26:09.751 INFO:scrape:Failure: Command failed on smithi061 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' 2020-12-29T14:26:09.751 INFO:scrape:8 jobs: ['5742882', '5742872', '5742874', '5742880', '5742857', '5742865', '5742863', '5742888'] 2020-12-29T14:26:09.752 INFO:scrape:suites intersection: ['1-ceph-install/mimic', '2-workload/{blogbench', '4-octopus', '5-msgr2', '6-final-workload/{blogbench', 'ec-rados-default', 'rados-snaps-few-objects', 'rados_api', 'rados_loadgenbig', 'rados_loadgenmix', 'rados_mon_thrash', 'rbd_cls', 'rbd_import_export', 'rgw', 'rgw_ragweed_check}', 'rgw_ragweed_prepare', 'start}', 'test_rbd_api', 'test_rbd_python}', 'ubuntu_latest}', 'upgrade:mimic-x/parallel/{0-cluster/{openstack'] 2020-12-29T14:26:09.752 INFO:scrape:suites union: ['1-ceph-install/mimic', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-workload/{blogbench', '3-upgrade-sequence/upgrade-all', '3-upgrade-sequence/upgrade-mon-osd-mds', '4-octopus', '5-msgr2', '6-final-workload/{blogbench', 'ec-rados-default', 'objectstore/bluestore-bitmap', 'objectstore/filestore-xfs', 'rados-snaps-few-objects', 'rados_api', 'rados_loadgenbig', 'rados_loadgenmix', 'rados_mon_thrash', 'rbd_cls', 'rbd_import_export', 'rgw', 'rgw_ragweed_check}', 'rgw_ragweed_prepare', 'start}', 'test_rbd_api', 'test_rbd_python}', 'ubuntu_latest}', 'upgrade:mimic-x/parallel/{0-cluster/{openstack'] 2020-12-29T14:26:09.753 INFO:scrape: 2020-12-29T14:26:09.754 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi ================================================================= info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/ logs: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/ failed: 8 dead: 0 running: 0 waiting: 0 queued: 0 passed: 8 Fail ================================================================= [5742857] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 01:11:15 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742857/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742857/ sentry: https://sentry.ceph.com/organizations/ceph/?query=af0bc4a356df44739c19589c10d06aca Command failed on smithi032 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5742863] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 01:07:38 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742863/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742863/ sentry: https://sentry.ceph.com/organizations/ceph/?query=44c08e6ff5b945b2b7ab3608d21fc47a Command failed on smithi137 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5742865] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 01:11:28 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742865/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742865/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2f764ff8185d44389aea2a79032e0fb5 Command failed on smithi193 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5742872] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 01:07:06 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742872/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742872/ sentry: https://sentry.ceph.com/organizations/ceph/?query=52eee31493df4398a8ff8e2486f76fbc Command failed on smithi114 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5742874] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 01:11:17 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742874/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742874/ sentry: https://sentry.ceph.com/organizations/ceph/?query=24f44a7c21084ef0bc261325af3bb750 Command failed on smithi047 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5742880] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 01:08:18 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742880/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742880/ sentry: https://sentry.ceph.com/organizations/ceph/?query=30f2a579a2c14082af507ec72cb0ccf1 Command failed on smithi151 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5742882] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-mon-osd-mds 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/bluestore-bitmap ubuntu_latest} ----------------------------------------------------------------- time: 01:10:25 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742882/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742882/ sentry: https://sentry.ceph.com/organizations/ceph/?query=425d4d35b1094a6cbd68f8ea7fb2528b Command failed on smithi061 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [5742888] upgrade:mimic-x/parallel/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-workload/{blogbench ec-rados-default rados_api rados_loadgenbig rgw_ragweed_prepare test_rbd_api test_rbd_python} 3-upgrade-sequence/upgrade-all 4-octopus 5-msgr2 6-final-workload/{blogbench rados-snaps-few-objects rados_loadgenmix rados_mon_thrash rbd_cls rbd_import_export rgw rgw_ragweed_check} objectstore/filestore-xfs ubuntu_latest} ----------------------------------------------------------------- time: 01:07:45 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742888/ log: http://qa-proxy.ceph.com/teuthology/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742888/ sentry: https://sentry.ceph.com/organizations/ceph/?query=52b50c74d6da4ffd8b9ce2aa4607ab68 Command failed on smithi077 with status 1: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' Pass ================================================================= [5742859] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 04:34:55 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742859/ [5742861] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 00:51:14 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742861/ [5742868] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 03:18:22 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742868/ [5742870] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 00:45:35 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742870/ [5742876] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 04:37:38 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742876/ [5742878] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/bluestore-bitmap thrashosds-health ubuntu_latest} time: 00:48:41 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742878/ [5742884] upgrade:mimic-x/stress-split/{0-cluster/{openstack start} 1-ceph-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-octopus 7-msgr2 8-final-workload/{rbd-python snaps-many-objects} objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 03:24:47 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742884/ [5742886] upgrade:mimic-x/stress-split-erasure-code/{0-cluster/{openstack start} 1-luminous-install/mimic 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-ec-workload 5-finish-upgrade 6-octopus 7-final-workload objectstore/filestore-xfs thrashosds-health ubuntu_latest} time: 00:51:44 info: http://pulpito.ceph.com/teuthology-2020-12-29_02:30:02-upgrade:mimic-x-octopus-distro-basic-smithi/5742886/